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ABSTRACT 


Navigation and terrain familiarity are critical for mission success in the military. 
Virtual environments (VEs) have often been suggested as a useful tool in addressing 
these issues. This thesis research addresses the utility of VEs to improve spatial 
knowledge of and navigation performance through natural terrain compared to traditional 
methods. In this experiment, fifteen subjects were assigned to one of three training 
conditions. The map group studied the environment using only an orienteering map. The 
real world group studied the environment using the map and explored the actual terrain. 
The VE group studied the terrain using both the map and a real-time VE. Measures were 
taken of both route and configuration knowledge. The results suggest four conclusions. 
First, training conditions have no statistically significant effect on an individual’s ability 
to obtain and demonstrate spatial knowledge of a natural environment. Second, spatial 
ability plays a significant role in navigation performance. Third, exposure to the actual 
terrain or to a virtual representation of the terrain seems to eliminate ambiguities in an 
individual’s mental map by providing dynamic imagery to clarify propositional 
knowledge gained from maps. However, this factor has not been shown to improve 
performance by the measures used here. Fourth, a high resolution 1:5,000 orienteering 
map provides extensive detail and consequently, navigation performance in this 
experiment is not likely to be indicative of performance using a conventional 1:24,000 


map. 
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I. INTRODUCTION 


A. PROBLEM STATEMENT 


The purpose of this thesis is to evaluate if training in a real-time high fidelity VE 
is superior in terms of spatial knowledge acquisition as compared to traditional military 
navigation or orienteering training methods (Chapter II, Section B.1). The research 
establishes a benchmark from which future research will continue to define the optimal 
level of fidelity and exposure times required in a vitual environment (VE) to provide 


spatial knowledge of a complex natural environment. 
B. MOTIVATION 
1. Important Applications for Spatial Knowledge Acquisition 


Spatial knowledge acquisition is important in a variety of applications and 
professions. We use our spatial knowledge of an environment to move freely throughout 
our everyday surroundings. Traditionally, we think of people using spatial knowledge of 
VEs when playing video games to quickly move through a virtual world. However, more 
vital applications of spatial knowledge can be seen in many areas. Taxi drivers use their 
spatial knowledge to transport passengers throughout the city. Fire fighters use spatial 
knowledge to conceptualize and maneuver through the environment where they are 


working even though the area is obscured by smoke and flames. 


In fact, any task that requires movement over a distance or through a complex 
environment demands some degree of spatial knowledge. Military personnel rely on 
their spatial knowledge of the environment to perform their missions. Soldiers use spatial 
knowledge of the environment to move rapidly and undetected to destroy the enemy. 
Special operations units utilize their spatial knowledge of an environment to quickly 
move through complex structures when conducting raids or hostage rescue missions. 
Other highly trained military units and personnel, such as Rangers or Scouts, use their 
spatial knowledge of an environment to conduct clandestine reconnaissance of an 
objective. They also utilize their spatial knowledge to establish ambush sites or secure 


critical terrain such as an airstrip. 


2. Proven Concept 


Human acquisition of spatial knowledge has been studied for many years. 
Thorndyke described a simple model for how humans acquire spatial knowledge 
(Chapter I, Section A) [THOR 80]. Since then, further research has been conducted in 
many areas which support Thorndyke’s model. In the domain of computer science, it has 
been shown that individuals can gain spatial knowledge of VEs [GILL 97] [RUDD 98] 
using a mental model comparable to Thorndyke’s. Similar research has indicated that 
detailed VEs can provide spatial knowledge transfer to real world environments [WITM 
95] [BLIS 97] [DARK 98]. 


3. Research Shortcomings 


Gaining spatial knowledge of a VE provides little benefit by itself. However, the 
transfer of knowledge to the real world does have many potential benefits. Individuals 
can train to perform hazardous tasks in the relatively safe surroundings of a virtual world 
prior to having to perform them in the actual setting. There are many questions about 
VEs that have not been answered in terms of how to facilitate a positive transfer of spatial 


knowledge to the natural environment. 


Computers allow us to view environments in numerous ways that are not 
physically possible in the real world. Viewpoints can be rapidly changed to provide 
individuals with alternative vantage points. We can examine the same object or location 
from any direction, plane, or altitude. We can also move through the environment at any 
speed we desire or merely teleport to the next location. However, the best combination 
of these capabilities to maximize spatial knowledge acquisition is still unknown. Studies 
focused on field of view, display devices, input devices, locomotion devices, and the 
navigation thought process are needed to find the optimal interface for acquiring and 


utilizing spatial knowledge. 


Much of the previous work has focused on increasing fidelity. However, little 
research has been conducted indicating that performance enhancement is _ linearly 
correlated to increasing fidelity. Flew experiments involve the study of how effective 
computer based training systems are compared to prior training techniques or how to 


better use what we already have. Recently, one research project indicated that an 


icrease in environmental fidelity does not necessarily translate into an increase in 
navigational performance [GOER 98]. Research in the relationship of model fidelity to 
performance is essential to identifying the minimal and optimal levels of detail required 
to obtain spatial knowledge of a specific environment. Such research will provide 
information needed to furnish model designers and builders with a template for the 


construction of virtual worlds. 
4. Army and DoD Relevance 


Defining the levels of model fidelity and system exposure times required to 
provide a positive transfer of knowledge to the real world can save the Department of 
Defense (DoD) time, money, and other valuable resources. With the decreasing DoD 
force structure, the increasing quantity and types of missions the military is asked to 
perform, and a shrinking military budget, commanders need faster methods to train their 
forces while maintaining or enhancing their abilities to successfully accomplish their 
missions. Computers provide commanders with the opportunity to conduct time- 
compressed training as they perform multiple mission runs changing parameters on the 
fly and reducing the wear and tear on the terrain and equipment. The major focus of 
computing and mission preparation tools should be to provide commanders with 
improved tools to maximize the quality and quantity of training that can be conducted 


during a limited time frame without introducing negative training effects. 


Virtual environments (VEs) may provide a cost-effective alternative to more 
traditional methods of training or mission preparation. The optimal computer training 
system has yet to be built for use in training dismounted forces. Even if an optimal 
interface and appropriate terrain model can be built, there is still the question of cost- 
effectiveness of such systems compared to traditional methods of model construction. In 
the past, the military has used mock-ups or other representations of an environment for 
mission preparation [GLIN 95] [FINN 97] [AMER 98]. Although these traditional 
model building methods are effective, they can be expensive, time consuming to 
construct, difficult to modify, labor intensive, and require large areas of terrain to be 


secured during construction and mission rehearsal (Chapter I, Section B.2). 


Little research has been conducted and published to validate the usefulness of 
VEs in providing spatial knowledge of natural terrain. If an individual can gain the same 
or only slightly better spatial knowledge using a VE over using a two-dimensional map, 
photo, or sketch of the environment, then it may not be cost-effective to utilize virtual 
representations of actual terrain or buildings to acquire spatial knowledge. It is possible 
that a VE may only be useful in training general navigational skills such as map reading, 
dead reckoning, terrain association, and route selection (Chapter II, Section C.1). Real- 
time VEs may be too complex to provide positive training transfer of spatial knowledge. 
If this is the case, a randomly generated VE may be more cost effective and just as 
efficient at training navigation skills as a virtual representation of an actual piece of 
terrain. Understanding the positive and negative effects of computer training and mission 
preparation systems will assist commanders in determining if and when they should use 


such systems: 


Understanding the actual and desired effects of computer training systems is 
essential for the development of a valid set of parameters for building models designed to 
equip users with spatial knowledge of an environment. This will assist the Army and 
DoD simulations personnel in determining when the construction and use of these models 
will be beneficial to mulitary forces. Such standards wil help to alleviate the 
misallocation of time, money, and computer assets for missions that do not warrant the 
use of computer generated models. The parameters will also assist in providing a 
common frame of reference for which these agencies, contractors, and programmers can 


consult with each other during the development and modification of such models. 
C. THESIS ORGANIZATION 


This thesis is organized in the following manner: Chapter II explores the 
background of spatial knowledge acquisition, the use of VEs to gain spatial knowledge, 
and land navigation and orienteering techniques. Chapter II outlines the development of 
the computer model and its capabilities as well as the methodology for this experiment. 
Chapter IV analyzes the data collected and discusses the results of the experiment. 
Chapter V provides the conclusions explaining the importance of the research and 


recommended areas for future research. 


There are seventeen appendices to this thesis that provide experimental outlines, 
listing of research materials, course layout, participant instructions, and raw data from the 
experiment. The appendices also discuss route complexities based on the International 
Specification for Orienteering Maps (ISOM) and computer generated routes, outlines 
environmental comparisons of training conditions, and describes the navigational thought 


cycle. 





Il. BACKGROUND AND PREVIOUS WORK 


A. SPATIAL KNOWLEDGE ACQUISITION 

Spatial knowledge or spatial cognition is a mental representation of a real or 
virtual environment [WICK 92]. Figure 2.1 graphically displays Thorndyke’s theory on 
how humans acquire spatial information to build a mental representation of their world 
[THOR 80]. In this model, the classifications of landmark, route, and survey knowledge 
are not mutually exclusive; knowledge at higher levels builds upon and augments 


knowledge gained from the preceding level(s). 





Figure 2.1. Navigation Knowledge 


Landmark knowledge is identified as the ability to recognize distinctive features 
associated with a specific location in the environment. This level of navigation 
knowledge is associated with the ability to store features, such as a specific hilltop or 
road intersection, in memory and recognizing it. Landmark knowledge is acquired 
through the direct observation of objects in the environment. It can also be gained 


through indirect observation of the objects in a medium such as a photograph. Successful 


landmark knowledge is demonstrated by the ability to recognize individual locations or 


unique objects within an environment [DARK 95] [THOR 80]. 


Route knowledge is defined as the procedural knowledge required to navigate 
along a route or path between landmarks or distant locations [GOLL 91]. It is derived 
from the ability to expand landmark knowledge into a larger, more complex arrangement 
of linked objects. Route knowledge is based on an egocentric (inside-out) viewpoint and 
is demonstrated by the ability to move from one landmark to another along a prescribed 
path. Route knowledge can be gained through repeated exposure to an environment, 
map, or through simulated exposure to the environment via a medium such as video 


[GOLD 82]. 


Finally, survey (or configurational) knowledge is the highest level of spatial 
knowledge. It represents a map—like or top down mental encoding of the environment 
and is based on an exocentric (outside-in) viewpoint. This last form of spatial knowledge 
is usually gained through map study but, can also be gained through extensive and 
repeated exposure to the environment [THOR 80]. Survey knowledge can be — 
demonstrated by an individual’s ability to describe the relative locations and the distances 
between landmarks or by devising new routes between landmarks even though the person 


has never traveled a route between them [BANK 97]. 
B. VIRTUAL ENVIRONMENTS 
1. Definitions 


Real-time graphics are defined as models which provide network delays of less 
than 0.1 seconds and can render images at a minimum of 8 to 10 frames per second for 
relatively static environments and up to 60 images per second for environments where 
objects have a high frequency of motion [DURL 95]. 

The classification of fidelity is more qualitative than quantitative since there are 
no metric scales which allow us to explicitly define levels of fidelity. For this research, 
the definition of a high fidelity walkthrough terrain model is a model that represents lines 
of sight and terrain masking, provides realistic depictions of the vegetation and structures, 


and can provide a real-time interactive environment for the user (Figures 2.2 and 2.3). 





Figure 2.2. Computer Model Figure 2.3. Actual Photo 


Landmark models are virtual representations of real world objects or locations 
that are easily identified, with defining characteristics, and are used by the participants as 
cues to navigate through the model. In Figures 2.2 and 2.3, the shack and poles are 
examples of landmark models. The term is devised for use in this thesis and derives from 


Thorndyke’s theory of navigation [THOR 80]. 


2. Model Usage for Mission Planing and Preparation 

For centuries, military units have conducted rehearsals in preparation for 
missions. Rehearsals were performed at all levels of command and took on many forms. 
Commanders utilized everything from checklists, sand table briefings, walk-through 
rehearsals using soldiers and open fields, video-taped runs of air corridors, to full fledged 
dress rehearsals utilizing complete mockups of the target area. The more extensive the 
rehearsal, the more resources and time were required. The more dangerous and complex 
the mission the more rehearsals were necessary for mission success. 

During World War II, the Army Air Corps prepared its pilots for air raid missions 
over Japan by showing them films of the precise routes they would be flying over enemy 
territory. These films were not produced from satellite imagery or continuing over flights 
by American reconnaissance aircraft. Instead, the films were produced by the Air Corps 
film and production unit stationed in Hollywood, California [AMER 98]. The production 
unit built a model of the Island of Japan using over fifty ten foot square platforms, tons of 
plywood, modeling clay, burlap, and paint. Using reconnaissance photos of the island, 
crews worked twenty-four hours a day for weeks, expending thousands of man-hours, to 
construct and paint an exacting replica of the island so that camera crews could film 
bombing routes for pilots. In order to maintain security, the model was built and filmed 


entirely on a single sound stage that was placed off limits to everyone except the 


personnel working on the project. Pilots routinely commented on how easy it was to 
recognize the terrain as they flew their missions because it was if they had been there 
before. 

In 1970, the United States Air Force planned a raid on North Vietnam to rescue 
American prisoners of war (POWs) from the Son Tay prison near Hanoi, North Vietnam 
[GLIN 95]. The mission was extremely risky. It required a task force of Air Force, 
Navy, and Army Special Forces personnel to travel over 340 miles across enemy 
territory, attack an enemy held compound, pull out the POWs, and fly back across enemy 
territory to the safety of American bases. To prepare the forces for the assault on the 
compound, a replica of the prison was constructed of two-by-fours and target cloth. The 
compound was built to exact dimensions to allow helicopter crews to practice taking out 
the guard towers with side mounted machineguns and land a single aircraft in the 
compound for the insertion of Special Forces soldiers and the extraction of the prisoners. 
Due to the nature of the mission, security was of the utmost importance. Even though the 
camp replica was built and rehearsals were conducted in Florida, thousands of miles from 
Southeast *sia, the model had to be dismantled daily to prevent detection by the Soviet 
Satellite Cosmos 335. Because of this, rehearsals using the mockup were conducted at 
night. Before morning, the model was dismantled and all holes were covered to ensure 
that no information could be gathered by the satellite and passed to the North 
Vietnamese. The ground team conducted over 170 trial runs through the camp replica 
before final authorization was granted to execute the mission. All this effort required 
over 148 personnel to support and execute the operation. Although the prisoners had 
been removed from the prison prior to the conduct of the raid, the task force was able to 
execute the plan to exacting precision without the loss of one American life. 

The Son Tay model was constructed with the assistance of an mconspicuous 
element of the CIA known as the Modeling Shop [FINN 97]. The three-dimensional 
modeling shop was located at the National Photographic Interpretation Center, in the 
Navy Yard. The section was established in 1964 to create three-dimensional models of 
key areas of the world to help planners and decision-makers with international policy 
decisions: The section operated for over thirty years and was finally replaced by 


computer generated modeling tools in 1997. Over the thirty-three years of the shop’s 


existence, the shop produced more than 862 products. These products include replicas of 
the U.S. Embassy in Iran and the Iranian Foreign Ministry, the Kremlin, sections of the 
city of Tehran, Russian aircraft, a section of Kuwait City from the U.S. Embassy in 
Kuwait to the Persian Gulf shoreline, and many other U.S. embassies throughout the 
world. 

The efforts to construct detailed models and maintain security for similar missions 
requires hundreds of thousands of man-hours and an untold number of assets. However, 
the need to conduct detailed rehearsals is essential for successful execution of any 


complex operation. 
3. Prior Studies of Spatial Knowledge and Virtual Environments 


In 1997, Bliss, et al studied the role of VE technology in acquiring spatial 
knowledge as outlined in Thorndyke’s model [BLIS 97]. They asserted that the spatial 
knowledge gained by navigating through a VE was comparable to the knowledge gained 
by navigating through the actual environment. They examined 30 firefighters performing 
simulated rescue operations in an office building. The firefighters were broken down into 
three study groups. The first group was given a map of the building to study, the second 
group was provided a VE, and the final group, the control group, was sent into the 
environment with no prior training. As they predicted, the VE and map study groups out 
performed the control group in the task of navigating through the structure. The 
researchers concluded that landmark, route, and survey knowledge can be acquired — 
through the use of a VE or map. Although the Bliss experiment’s performance measures 
indicated that VEs can provide landmark and route knowledge, there were no 
performance measures nor any qualitative analysis which suggested survey knowledge 
could be attained from VEs. 

Bliss’ research did not show any significant difference in performance between 
the VE group and the map only group. This begs the question of why we should use a 
VE to obtain spatial knowledge if a map provides the same or similar comprehension. 
This question is best addressed by a study conducted by Chase in 1983 [CHAS 83]. 
Chase examined individuals who studied maps of an environment which they had never 
been exposed to, and a second group of individuals who had extensive exposure to the 


environment but, never studied a map. His research suggested that the individuals with 
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extensive exposure to the environment had better landmark and route knowledge. 
However, the map study group had better survey knowledge of the environment. 
Therefore, Chase concluded that repeated exposure to an environment provides route and 
landmark knowledge but, this experience does not necessarily translate into increased 
survey knowledge. As stated earlier, survey knowledge is often obtained through 
exposure to an exocentric view while route and landmark knowledge are normally gained 
through repeated exposure to the environment. 

Wickens and Prevett showed that aviators who were exposed to an “immersive” 
viewpoint had better navigational performance over those who were given tethered or 
side views [WICK 95]. This is possibly due to the more natural representation or 
intuitive viewer interpretation based on the field of view [WICK 98]. Taking this into 
account along with the Chase experiment, it 1s possible that the combination of map study 
and VE exposure may provide the optimal solution for providing total spatial knowledge 
of an environment [CHAS 83]. 

Banker showed that individuals with intermediate land navigation skills could 
gain and transfer their spatial knowledge from a VE to a natural environment [BANK 
97]. He studied three groups; a map only group, a map and non real-time VE group, and 
a control group who studied a map and the actual environment. None of the participants 
had any prior knowledge of the course, terrain, or map. Participants were give one hour 
to study the environment and plot their route through the environment before the map 
was taken away from them and they were required to navigate the real world course. 
With only a one-hour exposure time, Banker discovered that the VE showed a significant 
increase in performance for intermediate navigators only. He concluded that due to 
advanced navigators’ abilities to extract vast amounts of information from the map, the 
VE was of no additional advantages to them. He further surmised that beginning 
navigators had reached information overload with all the materials provided and they 
were unable to separate the noise from the essential information. 

Goerger, et al conducted a similar study to the Banker experiment using a 
complex man made environment [GOER 98]. They compared two groups; a map only 
study group to a map and VE study group. None of the participants had any prior 


exposure to the building or the floor plans. Each group was given thirty minutes to study 


floor plans of the seven-story structure and a clue sheet (Figure 3.17). The VE group was 
also exposed to a high fidelity, real-time computer representation of the building during 
the thirty-minute study phase. The floor plans had the control points and a designated 
route marked in red. After the thirty-minute study phase, the floor plans were taken from 
the participants who were then escorted to the building for the testing phase. The results 
of the experiment showed the map only group significantly outperformed the VE group. 
The researchers concluded this was due to the short exposure time. They surmised the 
limited exposure time did not allow the VE group to resolve the exocentric differences 
between the, floor plans and the virtual world. Furthermore, this did not allow 
participants to translate their knowledge of the environment from the maps to the VE to 
the real world. This indicates that performance on spatial knowledge tasks after brief 
exposure to a high fidelity, real-tume VE does not always exceed results gained from 
traditional navigation training techniques [GOER 98]. 

Although it has been demonstrated that virtual worlds can be effective in 
providing some level of spatial knowledge, we are far from understanding which 
. characteristics of a VE and its interface are most effective. A study by Witmer, et al 
investigates the issue of user interface [WITM 95]. They examined 64 college students 
navigating a large building. The participants were broken into three groups; those who 
received verbal directions on how to navigate through the building, those who explored a 
VE of the building, and those who explored the actual building. Results indicated that 
the real world participants outperformed the VE participants who in turn did better than 
the verbal study students. Although the VE group outperformed the verbal group, some 
VE participants did not perform well. Further analysis of the training data indicated that 
VE participants who became entangled in the model due to difficulties with the interface 
spent much of their time bouncing into walls, becoming disoriented, and subsequently 
made many wrong turns. These same individuals had difficulty navigating through the 
actual building. Although these are qualitative measures, the data indicates that a poor 
interface can lead to disorientation and diminish the overall training efficacy of the VE. 

In other research, Williams, et al investigated active versus passive control during 
flight mission preparation [WILL 95]. The study compared individuals who passively 
observed a flight through a virtual world to those who actively controlled the flight of the 


aircraft through the high fidelity flight simulator. The study indicated that those 
participants who controlled the rehearsal phase outperformed those who passively viewed 
the training flight. They concluded that the optimal VE designed to enhance spatial 
awareness would include active control by the user. 

Based on these studies we can conclude that VEs are useful in providing 
landmark and route knowledge. Augmented with map study to provide survey 
knowledge, the two media can provide a powerful method for obtaining spatial 
knowledge of unfamiliar environments. To optimize the use of a VE, the interface must 
be transparent to the user while providing active control of the viewpoint. Optimal 


exposure durations and fidelity levels are still a questionable element of the VE. 


4. Model Classifications 


The first aspect that must be considered before building any model is its purpose. 
Requirements for building a three-dimensional architect’s rendition of a building’s 
blueprints and the requirements for building a real-time walk through model of the same 
structure are significantly different. The first may require only a wire frame or flat 
shaded polygons. The latter may require expanded coloring schemes and texturing to 
provide a realistic appearance. The architect’s rendition may not be concerned with 
collision detection, the portrayal of furniture, sound effects, or independent moving 
entities in its VE, while the real-time walkthrough model may. All these factors 
determine the system requirements, the type and quantity of environment ~:ormation 
required, the programming language, the tools, the file structure, and the procedural steps 


for the construction of the model. 


Virtual environments have developed beyond the initial desert and overflig.. 
models produced in the early 1970’s through the 1980’s for the Department of Defense. 
Today’s models have extended into the realm of complex natural environments. 
Throughout this thesis, the term “complex natural environment” is utilized. This 
classification of topography represents a piece of terrain at least one square kilometer in 
size which consists of vegetation and elevation changes that mask lines of sight and 
provide obstacles to cross country movement. To facilitate navigation, the terrain must 


have numerous distinctive landmarks. Finally, it possesses numerous paths, trails, or 


roads that provide opportunities for parallel navigation errors (Chapter IV, Section 


A.4.a). 


Terrain models can be separated into four distinct categories based on the mode of 
travel through the environment. The four model categories are Dismounted Movement, 
Ground Vehicles, Rotary Wing Aircraft, and Fixed Wing Aircraft (Figure 2.4) 
[SULL 98]. Each of these categories requires a lucid level of fidelity or combinations of 


levels-of-detail (LODs) that must be considered before the start of model construction. 
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Figure 2.4. Fidelity vs Mode of Travel 


The first category is the Fixed Wing Aircraft Model. Jet aircraft simulators and overhead 
“God’s eye” view are in this category of model that requires limited detail due to the 
speed and altitude of the viewer. Models of this category typically consist of aerial 
photos or satellite imagery placed over elevation data. Distinguishing landmarks such as 
cities, lakes, major roads, valleys, and large landmasses facilitate navigation through this 


type of environment. 
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The Rotary Wing Aircraft Model is used in helicopter trainers and “pop-up” 
views. This model requires enough detail to distinguish groups of trees and buildings, as 
speeds are reduced to less than 100-mph and altitudes less then 200 feet. A model of this 
type will combine aspects of models from the higher resolution category Ground Vehicle 
Models with those of the lower resolution Fixed Wing Aircraft Model. Levels-of-detail 
(Chapter III, Section C) are used to smoothly page models in and out as the point of view 
moves between modes of travel. Navigating through this type of environment is 
facilitated by the user’s increased ability to distinguish landmarks such as road and river 
intersections, small lakes, city blocks, draws, ridges, and hill masses. Additional items 
such as power and telephone lines, side roads, and major landmarks must be modeled in 
this environment. This increased fidelity assists in navigation and in providing the detail 


required allowing the aircraft to maneuver through the terrain in a realistic manner. 


Surface level perspectives begin with the Ground Vehicle Model. Lines of sight 
must be preserved in order to provide the illusion the viewer in moving through the 
terrain. Realistic masking and unmasking of natural and man made landmarks are key to 
' providing a representative model. This model also requires a higher level of detail as 
actual three-dimensional objects, such as buildings, statues, bridges, and signs are 
represented and distinguishable to the degree where they can be utilized as landmarks and 
navigational aids. General textures may be utilized to provide increased detail to the 
buildings and terrain providing a more realistic illustration. At speeds greater than ten 
miles an hour, texture fidelity can be reduced since actual details are blurred. due to the 
velocity of the camera. This is based on the understanding that as views are overloaded 
with information or exposed to a high-density display, they filter the scene based on 
global and local perception [OLSO 70]. In such an environment, the individual will 
focus on objects which are unique or out of place with regard to the surrounding 


environment [FRIE 81]. 


The last category is the Dismounted Movement Model. This category requires 
the greatest level of detail of the four. The limited speeds of the individual moving 
through the model and the approximation of the individual to the objects in the model are 
the primary factors for increased resolution. In a fly or drive through scenario, the level 


of the trees and buildings, or the boundaries of roads, trails and clearings are the 
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customary limits of the camera. In a walkthrough environment, the user may need to 
infiltrate through the low ground or riverbeds, seek concealment in a tree line, pursue 
cover in a ditch or trench, or setup a patrol base in a thicket. Thus, much more 
knowledge of the actual environment is required and a greater level of detail must be 
represented. This is the foundation on which the model for this experiment was 


developed (Chapter II, Section C.). 


Due to the general nature of the categories, the ever-increasing capabilities of 
computer hardware, and the expanding demands of the user, the borders between these 
categories are blurred and in many cases overlap. Further research must be conducted to 
solidify these boundaries and establish a set of general specifications to assist model 
developers and clients in determining what type of model is required. This can only be 
done through human experimentation. As computers and the virtual modeling 
community continue to move forward, the boundaries between these categories need not 
continue to shift towards increased fidelity requirements. This model can be solidified if 
category specifications are standardized based on complete understanding of user task 


requirements. 


C. LAND NAVIGATION AND ORIENTEERING 
1. Military Land Navigation 


Efficient, well directed navigation is the process of moving through an 
environment in a manner in which the individual knows the start position, current 
position, destination, route to travel, and distance traversed. Navigation requires 
knowledge of location, direction, and destination, while having a means of travel through 
the environment [WICK 92]. It is also an evolving process. Navigation not only 
involves acquiring knowledge of and strategies to move through a space, it also requires 
modification of this metaknowledge of the environment as we move through the space 
and identify changes or inconsistencies with our mental representation [JUL 97]. 
Navigation can occur in, on, or through many different media such as land, sea, air, and 
space. 

For military personnel, navigation plays a fundamental role in nearly any mission. 


For Marine and Army personnel, this is even more apparent. The Department of the 


le 


Army (DA) has recognized the importance of map reading and navigation skills for its 
personnel and has woven training requirements into its latest doctrine [FM21 93]. 

The Army implements a building block approach to training its people in 
navigation. Training Starts at the initial indoctrination of personnel and continues at unit 
and Department of the Army training facilities. As soldiers move up the chain of 
command, the level of required expertise in navigation and map reading skills increases. 
Therefore, the DA has initiated land navigation in its basic training courses to ensure 
soldiers have the minimal skills for basic map reading and dead reckoning. Additional 
education occurs at the initial and intermediate leadership courses to refine and enhance 
navigation competence. 

With a basic proficiency of map reading and dead reckoning, the Army feels their 
soldiers have the navigational foundation required to perform as team members during 
cross-country movement. However, military leaders require additional training that 
focuses on route selection and tactical movement techniques. These skills are trained at 
the unit and during basic leadership courses. Finally, the military also implements a more 
intense level of navigation and map reading instruction at the intermediate leadership and 
staff schools. This final level of navigation training is designed to ensure that mid level 
leaders and staff officers have the increased abilities to conduct detailed analysis of an 
area of operations. These personnel are expected to glean information from military 
maps, non-military maps, and aerial photos to develop an accurate and timely analysis of 


any area. 


Basic skills taught in military land navigation training include map reading and 
use of an M2 or lensatic compass. Other navigational tools and field expedient methods 
for determining cardinal directions such as watch, star, and shadow-tip are also taught. 
Even the use of a global positioning system (GPS) for determining location has become 
an essential part of the curriculum. For cross-country movement, the military focuses on 
two basic land navigation techniques; dead reckoning and terrain association. These 


skills are perishable and must be continuously practiced to maintain proficiency. 


Dead reckoning is the ability to navigate through terrain without the use of trails 
or intermediate landmarks. It consists of two fundamental steps. The first step consists 


of determining the direction of movement and the distance to travel. The second step 
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incorporates traversing the terrain utilizing a mechanism to determine direction of 
movement and a device to record the distance covered. This technique of navigation is 
normally associated with dismounted cross-country travel during limited visibility or 


through thickly vegetated environments. 


The most difficult and valuable capability in military navigation is terrain 
association. This skill is used to visualize the features of the map and correlate them with 
the actual terrain features. The process of terrain association starts with aligning the map 
to the terrain. The second step is to determine one’s location. This step is key to 
successful navigation. With knowledge of one’s current location, a navigator can then 
determine the distance and direction to the destination. Without knowledge of one’s 
location, the individual is doomed to roam aimlessly around the terrain until a fix is 


obtained. 


Terrain association is also more forgiving than dead reckoning. Errors from 
terrain association can easily be resolved as the navigator reestablishes location and 
adjusts direction of travel. Errors with dead reckoning may not be discovered until the 
navigator has reached the prescribed distance. Then the actual location must be 
ascertained before a new direction of travel and distance to the intended destination can 
be determined. While utilizing terrain association, a navigator will focus on prominent 
terrain features, such as hilltops to guide direction of travel. The navigator may also use 
handrails, such as rivers or ridgelines to guide movement. Handrails are linear terrain 
features such as a river, road, trail, power line, or ridge that is parallel or congruent with 


the desired or most direct route that an individual follows as a guide [FM21 93]. 


Finally, to prevent over shooting the objective, a navigator utilizing terrain 
association will use a catching feature, such as a stream, to act as a limit of advance, 
stopping movement in a particular direction. A catching feature is a prominent piece of 
terrain used by navigators to indicate to the need to change direction or stop movement 


[FM21 93]. 


To assist with training and honing navigational skills, the military runs land 
navigation and orienteering courses. Orienteering is a competitive sport that combines 


land navigation and cross-country running. The Army divides orienteering into four 
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categories: route orienteering, line orienteering, cross-country orienteering, and score 
Orienteering. Route orienteering is used for novice navigators. Soldiers follow behind a 
guide who takes them through the course. Soldiers are required to trace the route on their 
maps and circle the location of control points located along the route. Line orienteering 
consists of copying a pre-designated route onto one’s own map. Soldiers then follow the 
prescribed route and circle the location of controls located along the route. At the 
completion of a route or line course, maps are compared to a master map for accuracy. 
Cross-country and score orienteering are two of the most common forms of what is 


known in the broader category of sport orienteering. 
2. Sport Orienteering 


Sport orienteering involves navigating through an environment utilizing a map, 
clue sheet (Figure 3.10), score card, and compass in order to find a series of three sided 
markers known as controls. Each face of the control marker is usually 12” by 12” and 
colored half international orange and half white (Figure 2.5). Barring no obstacles to 
observation, control pomt markers are designed to provide a participant with a. 


recognizable view no matter which direction the contro] point is approached [LOWR 89]. 





Figure 2.5. Contro] Marker 


Most orienteering events are established in natural environments with multiple 
courses that provide varying degrees of difficulty. The length and the location of the 
contro] points determine the difficulty level of a course. The further apart and more 


technically demanding the placement of the contro! points, the higher the overall rating of 
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the course. Course levels are broken down by ability groups from beginner, advanced 
beginner, intermediate, and advanced competitors and are identified by a color-rating 
scheme. A white course, or beginner course, is typically 3km straight-line distance and is 
comprised of anywhere from seven to fifteen control points. An orange course, or 
advanced beginner course, 1s usually slightly longer with control points placed in more 
restrictive terrain than the white course. 

Contestants are allowed to plan their own routes through the course based on their 
skills and experience. Some courses require competitors to find the controls in order 
(cross-country orienteering) while others allow contestants to find the controls in any 
order the participant desires (score orienteering) [FM21 93]. A circle on the map 
indicates the locations of the controls. The circles normally cover a 30 to 50 meter area 
and are further defined by descriptions on a clue sheet. An individual proves that they 
have visited a control by “punching” his or her score card with a punch which produces a 
distinctive pattern on the score card [BANK 97]. A combination of shortest time and 
number of control points found score the event. Each control point has a weight assigned 
to it based on its technical difficulty. The individual with the highest point total and 
fastest finish time 1s the winner. 

Land navigation training affords researchers the ability to provide an adequate 
level of spatial knowledge through the use of proven training methods while 
implementing new training devices. Sport orienteering furnishes the testing platform for 
studying the spatial knowledge of individuals who have undergone land navigation 


training. The combination of the two makes for an excellent research opportunity. 
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Hl. METHODOLOGY 
A. TEST ENVIRONMENT 

A combination of military navigation and sports orienteering was used in the 
development of the methodology of this experiment. The course length, difficulty level, 
clue sheet, and marking system are comparable to an average orienteering orange course 
(Chapter I, Section C.2). The use of terminology, reliance on memory skills, and 
navigation techniques are more closely related to military land navigation. 

The orienteering course is established on a 1.2km by .7km piece of terrain.’ The 
terrain is located on the central coast of California in the former training area of the 
recently closed Fort Ord. The terrain is populated with outhouse facilities, shacks, 
pavilions, telephone lines, and a criss-crosseded trail and road network. Elevation varies 
from 90m to 123.4m above sea level. The limited yet distinctive changes in elevation 
provided a course that was neither a test of athletic ability nor a flat featureless 
environment. 

The vegetation on the terrain can be broken down into three distinct categories; 
perennial grasslands, oak forest, and maritime chaparral. Perennial grasslands cover 
about one fifth of the course and are characterized by knee to waist high grasses and 
some widely scattered oak trees and underbrush. These areas have excellent visibility 
and possess limited obstacles to cross-country movement. The second category is the oak 
forest that makes up nearly forty percent of the course. This terrain is populated with 
inland or coast live oaks that vary in height from 25 to 45 feet. Inland oaks often have 
canopies that reach all the way to the ground creating mobility as well as visibility 
obstacles. The woodland varies in density and undergrowth allowing some areas to be 
easily traversed. Other areas are thick with vegetation or contain large quantities of 
poison oak making cross-country traversal more difficult. 

The final category is maritime chaparral, which makes up forty percent of the 
course. Chaparral tends to grow in dense uniform thickets and has abrasive 
characteristics which create a considerable barrier to cross-country movement. Since 


cross-country movement through these areas is not recommended, walking around, as 


" The dimensions of New York City’s Central Park are approximately 0.9km by 4.2km which is 
four times the size of this model [MAP 98}. 


pus 


opposed to attempting to battle through, is the best way to negotiate these areas. 

Due to its use as an active duty Army installation for much of this century, the 
area is criss-crosseded with a network of paths, trails, and minor roads. In conjunction 
with the occasional man-made structures listed above, the trails allow for the creation of a 
course rich in landmarks with many opportunities for the navigator to make parallel and 
murror errors [BANK 97]. Parallel errors occur when individuals are not on their planned 
route or desired location but, on a route that runs parallel to the intended route or similar 
location within the environment. This occurs because of the ambiguities of the 
environment, which allow individuals to make mistakes without realizing their errors. A 
mirror error happens when a participant reaches a decision point, is faced with multiple 
options, and chooses the wrong one. For example, the participant comes to a fork in the 
road where he originally planed to take the right fork but, instead takes the left fork. 

B. MAP AND COURSE DEVELOPMENT 

MAJ William Banker utilizing the ISOM [INTE 90] developed the course map 
(Appendix F.3) [BANK 97]. Banker used a 1993 digitized aerial photograph of Fort Ord 
to provide the general outline of the environment. He also conducted ground 
reconnaissance to categorize the terrain and identify features not visible in the photo. 
The map was field checked by MAJ Banker in May of 1997 and by CPT Goerger in May 
of 1998. The scale of the map is 1:5,000. Traditional military operations maps use 
1:25,000 or 1:50,000 scale maps. Competition orienteering maps are usually produced at 
a scale of 1:15,000. The higher resolution map for this course provided an extremely 
accurate depiction of the terrain. Trails and features down to two meters in diameter 
could be represented on this map. This afforded participants numerous landmarks and 
paths from which to plan their routes. The unconventional scale of the map may cause 
problems with discerning distance while the orienteering symbols and terrain features 
may cause participants unfamiliar with such terminology some confusion. To assist 
participants in overcoming these problems, a legend describing each symbol and color 
code as well as a distance scale are attached to the top of the map. 

The course begins at the intersection of Gigling and Watkin’s Gate roads and 
extends to the southwest. It consists of a starting point and nine control points. The 


control points were placed in accordance with the standards for a traditional orienteering 
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orange course. This means none of the control points are located on trails or roads. This 
ensures that participants will not merely stumble across control points and requires all 
participants to conduct some cross-country movement. The placement of control points 
allows for numerous routes to each location while limiting the possibilities for 
revisitation of previously traveled routes [BANK 97]. No extraneous or false control 
points are located on the course. | 

Participants are required to navigate through the control points in order. The 
straight-line distance between the control points is 2070m. Planning a very conservative 
route which sticks as close as possible to the roads and trails, a participant can plan a 
route in excess of 4560m or 2.85 miles. The course is designed to test memory and 
navigation skills while limiting the physical skills required to complete the course. 
C. MODEL DEVELOPMENT 

The model is a real time replica of the test environment and was developed based 
on the aerial photograph, course map, and ground reconnaissance. It was created on a 


SiliconGraphics Industry’s (SGI) Onyx Reality Engine-2 workstation (Table 3.1). 










Table 3.1. Machine Characteristics 


While moving through the model at six to ten miles an hour, the system generates 
34,360 plus triangles at 10Hz utilizing only one processor.” The model was developed 


on an SGI graphics workstation in anticipation that within two to three years, graphics 


* The terrain model with all attachments consist of over 50,850 polygons (18,336 terrain polygons; 
14,690 billboards at two polygons each; 65 structure models at an average of forty polygons each; two 
vehicles at 50 polygons each; 77 forest walls at two polygons each; 144 (+) roads, trails, paths, & trench 
line sections at two polygons each). 
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workstations costing ten to twenty thousand dollars and possible high end personal 
computers will be able to run a model of similar complexity at comparable rates of 
performance. 

OpenGL and IrisGL are the substructure languages for some comprehensive 
development tools such as Multigen and Corypheaus's Designer Workbench, 
EasyTerrain, and EasyScene. A combination of Multigen and the Corypheaus tools were 
used in the development of the model. 

Dismounted movement models [SULL 98], which replicate cross-country 
movement through rough terrain, require elevation accuracy to the nearest meter or better 
to represent ditches and holes. Model elevations are acquired from Digital Terrain 
Elevation Data (DTED) repositories. The experiment’s model consists of DTED-2 data 
modified using Microsoft Excel to create DTED-5, one elevation post every meter. As 
part of his research in model building in 1997, MAJ William Banker made these 
modifications. | 

DTED is produced and distributed by the Department of Defense's National 
Imagery and Mapping Agency (NIMA) and comes in Level 0 through Level 5. DTED 
Level 1 (DTED-1) is one elevation post for every 100 meters and Level 2 DTED (DTED- 
2) is one elevation post for every 30 meters. Both DTED-1 and DTED-2 are 90 percent 
accurate to +/- 30 meters. Levels * - 5 are classified and only available to qualifying 
agencies and personnel. DTED-1 i: available to the general public for the continental 
U.S., Alaska, Hawau, Puerto Rico, and the Virgin Islands [EART 95]. 

Levels-of-detail (LODs) are utilized to specify at what resolution a section of 
terrain is rendered. When a model is created, sectors are established to help with the 
computer's culling process by identify what items should be displayed with respect to the 
location of the viewpoint. Sectors are established in a database as rectangular sections of 
terrain where the size of the sectors is based on the distances established for the LODs. 
These sectors are tiled together to create a grid. The sector where the viewpoint is 
located is displayed at the highest level of resolution and adjacent sectors are displayed at 
the next level of resolution. The further the sector is away from the point of view, the 


lower its level of resolution (Figure 3.1). 
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¢ - Point of View (POV) 


Th - Radius of Sector 


D» - Distance from the 
POV to the Sector’s 
center 





Figure 3.1. LOD Sector Grid 
As the viewpoint moves through the grid, sector resolution levels will switch to 
accommodate the new viewpoint position [VOGT 97]. Adjacent sectors are switched to 
the highest level of resolution when the distance between the point of view and the center 


of mass of the next sector (D2) is equal to, or smaller than, the distance between the 
center of mass of a sector and one of its corners (Dj). The distance for LODs was 


determined utilizing the formula for the resolution angle of an object. This angle is based 


on an object’s distance from the participant thus, [distance to the object = (size of the 


object / 2) / tan (resolution angle / 2)] [SCHI 82] [SPER 97].° On angles less then 10°, 
Schiffman asserts that the visual angle does not need to be divided in half. The revised 
equation is [distance to the object = (size of the object) / tan (resolution angle)]. 

O'Kane illustrated the stages of target acquisition based on Johnson's bar pattern 
methodology [OKAN 95] [JOHN 58]. She describes how a potential target goes through 
four phases: find, detect, recognize, and identify. The last three phases have application 
in the graphics world. Detection is when an object can be discerned from its background, 
for example; there is an object on the hill. As we draw closer to the object it is 


recognized as its features are discernable enough to place it in a specific category; 1.e. the 


> Schiffman explains how to determine the visual angle of an object based on the size of and 
distance to the object. He uses the formula [tan (visual angle/2) = (size/(distance/2))]. Sperber states in his 
notes that the resolution angle, the angle at which two similar items are no longer distinguishable as 
different objects, is 1/60" of a degree or less. 
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object on the hill is a truck. Finally, we approach close enough to the object that we can 
identify it by resolving its unique features; i.e. the truck is a Ford F150. The closer we 
get to the object the greater the size of the object’s visible angle is in our field of view. 
These distances and angles can be calculated using Johnson's bar pattern methodology 
[JOHN 58]. For more common military targets, the angles for detection, recognition, and 
identity have been calculated using Johnson's bar pattern method and are listed in mrads* 
in Dudzik's Electro-Optical Systems Design, Analysis, and Testing [DUDZ 93]. 

Utilizing the above formula and Dudzik's table for resolution factors for a soldier, 
distances were calculated for identification, recognition, and detection distance. The 
average soldier standing in open terrain on a clear day can be identified up to 234m. The 
same soldier is recognized as a human up to 492m away and can be detected at distances 
up to 1247m with the naked eye.” 

The model has three LODs. The first change over point is placed at 150m, the 
second at 450m, and the final LOD extends out to infinity. If the model were larger, the 
third LOD change over point could have been placed at 2000m or the far clipping plane 
- could be set at this limit. The LODs for this model were based on the abilities of the 
computer system to handle the model fluidly, the previously stated calculations, and the 
standard distances used in Army marksmanship. As discussed earlier, individual target 
silhouettes lose their identity at 230m. Humans are unrecognizable at 450m to 500m and 
are not detectable at distances greater than 1250m when viewed by the naked eye. The 
average rifleman classifies targets within 100m as close targets. Starting at 150m, targets 
are considered midrange [FM23 89]. Army sniper training routinely works within the 
1000m range even with a 10x scope [FM23 94].° Utilizing the 450m (diagonal 636m) 
mark as the limit of recognition of a human, this was established as the switching point 
for the second and third LODs. Attempting to maintain square sectors and using the max 
range of close targets, the switching point for the first and second LODs was placed at 

4 1 mrad equals 5.72958 x 10° degrees [BEYE 84]. 

> The same relative distances can be calculated using a combination of Schiffman’s formula [SCHI 
82], Sperber’s resolution angle, and the MIL-STD-1472D’s [SPER 89] 95" percentile measurements. For 
the average adult male’s interpupillar breadth (7.1cm), head height (14.5cm), and chest breadth (36.7cm), 


the average individual will not be able to distinguish a pair of human eyes at 244m, a head on a pair of 
shoulders at 498m, and the human stature at 1261m. 


© When estimating range, a sniper will determine the distance to a 6-foot man measuring two mils 
as 1000m [FM23 94]. 
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150m (diagonal 212m) (Figure 3.2). This created sectors which were 300m x 300m and 
placed the switching point for the last LOD at 750m (diagonal 1060m). 


Near Clipping Plane 
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Figure 3.2. LOD Distances 
Once the digital terrain data was converted into a terrain model and LODs, the 
model's surface was colored with earth tones reflective of the actual terrain. A black and 
white aerial photo texture was then attached to the terrain (Figure 3.3). This texture map 
was used to place roads and buildings and to define the boundaries of the wooded areas. 
The black and white photo was left on the final model to emulate shadow effects. The 
black and white photo allowed the base color of the model to bleed through the photo and 


give the terrain a more natural appearance. 





Figure 3.3. Elevation Model with Black and White Aerial Photo 


After the terrain base was generated, landmark models were constructed and used 
to populate the terrain’s surface. Landmark models are replications of structures that are 
distinctive and easily identifiable in the environment. Using photos and measurements of 


the structure, realistic replicas of the structure's exterior were developed. Telephone 
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poles, two small shacks, five individual outhouses, two distinct pavilions, two distinct 
ypes of cement pads, three separate rock piles, and sandbags were developed and used as 
landmark models. The aerial photo helped maintain alignment and object orientation 
when placing landmark models onto the terrain model. 

Next, linear objects such as roads, trails, ditches, and power lines were placed. 
These linear objects helped establish boundaries and preserve proportions. With 
boundaries clearly defined, vegetation was added. Since most vegetation is complex in 
structure and liberally distributed, billboard textures (Figures 3.4 & 3.5) were utilized. 
The use of billboards greatly reduced the computational requirements for rendering the 
wooded areas. The billboards rotate with the movement of the camera, always providing 
the user with a perpendicular view of the object. Billboards provide the illusion of three- 


dimensional objects. 





Figure 3.4. Textured Billboards Figure 3.5. Untextured Billboards 

Four distinct trees, three types of under brush, and two bushes were utilized in the 
model (Figure 3.6). An essential aspect of creating realistic billboards is the use of 
textures. Editing textures is more of an art than a science. Although there are many 
techniques for editing images, a commonly used methodology is to adjust colors and light 
levels, alter perspectives, correct image impurities, crop, size, save in an appropriate 
format, and final editing. These steps were completed utilizing Adobe Photoshop and 
MultiGen. The same textures used in the construction of the brush billboards were used 
in the development of walls that were intermixed with the billboards to provide the 


appearance of a forest. 
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Figure 3.6. Trees, Bushes, and Walls 


The four tree billboards were randomly placed based on their size, largest to 
smallest, at a ratio of 40, 30, 20, and 10 percent respectively. The three distinct types of 
brush and two bushes were evenly mixed at 20 percent each. The three brush textures 
were also used to create three distinct wall types to represent forests at distances greater 
then 450 meters. A thick forest, approximately 25 percent of the model, has a dispersal 
of 15,000 trees and 10,000 undergrowth billboards per square kilometer. Moderate 
forests, 20 percent of the model, had 15,000 trees and 7,000 pieces of undergrowth per 
square kilometer. Forests with little to no undergrowth make up nearly 35 percent of the 
model and are covered with 7,000 trees and 500 pieces of undergrowth per square 
kilometer. About 10 percent of the model is covered with undergrowth at 10,000 plants 
per square kilometer. The rest of the terrain is a mixture of trails and open ground 
(Figure 3.7). In total, the model uses 34 textures and 9 billboards to populate the 
environment. The final model contains: twenty-two telephone poles, two shacks, five 
outhouses, three pavilions, nine cement pads, three rock piles, two sand traps, a paved 
road, over 9,960m of dirt roads, trails, and paths, two trench lines, 200 plus sandbags, 77 
forest walls, and over 14,690 billboard trees, bushes, and brush. Two military trucks, 
(HMMWV) were also added to the model. These vehicles were used to provide a 
common object as a reference to resolve issues of size and perspective (Chapter IV, 


Section B.12). 
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Figure 3.7. Model Populated with Objects 
The LODs were determined for movement along the ground at a camera height of 
five feet eight inches. However, if a participant decides to travel along the terrain at an 
elevated level, he will experience a “popping” in and out of forest walls (Figure 3.8) and 
trees (Figure 3.9) as he crosses an LOD switch over point. Programmers use LODs to 
render closer objects at a level of higher detail than objects at greater distance. This is 
done by instructing the system to swap in images and structures with greater detail for 


closer objects and structures of lesser detail for further objects. 





Figure 3.9. Trees Sketch 
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Depending on the distance and height of the viewpoint, the exchange of walls for 
independent billboard is visible and is commonly referred to as “popping”. To alleviate 
this popping, the model may require dynamic LODs that change depending on the 
camera’s distance above the ground. Additional LODs can be added which allow the 
modeler to slowly thin the trees out at different distances and makes the popping more 
understated. These options can be computationally expensive. To reduce model 
complexity, these features were not implemented in this model. 

When viewing forest walls from an elevated position, the user can gain a false 
sense of vegetation density because of the lack of vegetation behind the walls (Figure 
3.10). Placing a canopy over the top of the forest walls to represent wooded terrain can 
curtail this misperception. To reduce the amount of texture memory used for this model, 


forest canopies were not implemented in this model. 





Figure 3.10. Elevated Wall Sketch 


Once the terrain, structures, and vegetation were assembled and in place, the basic 
model was complete and the interface was developed. Utilizing the Coryphaeus 
EasyScene and API tools, an interface was developed to allow the model to be explored 
and the environmental conditions to be modified (Appendix P). Utilizing the BG 
Systems FlyBox (Figure 3.11) and a standard 124 key keyboard, participants were able to 
move through the environment at speeds of up to 10 miles an hour with viewpoints fixed 
at five feet eight inches or fifteen meters above the ground. Environmental conditions 


could be set for one of six conditions: sunny, cloudy, stormy, dawn, dusk, or midnight. 
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Figure 3.11. BG Systems FlyBox 

The BG System FlyBox joystick is used as the main interface because it provides 
the participant with a compact set of instruments that can be manipulated utilizing only 
two hands without rec.iring an individual to have the dexterity required to use a 
keyboard. The joystick provides six degrees of freedom. This allows the participant to 
maneuver the viewpoint forward and back using a side lever and orient the camera’s pitch 
and rotation as well as the movement heading using the joystick. The automatic 
centering feature of the joystick ensures the aligning of an individual’s field of view with 
the direction of movement when a participant releases the controls. 

Participants can teleport the viewpoint to any of the nine checkpoints. If 
navigational assistance is needed the interface allows the user to have a top down view of 
the model from 1500 meters or a compass is displayed indicating direction of travel. If 
the compass is activated, all linear movement of the viewpoint is stopped, however the 
individual can still turn or pitch the camera view. The FlyBox joystick 1s programmed to 
allow an individual to turn the camera view left and right up to 90 degrees and pitch the 
camera view up or down between 0 and 23 degrees to see the skyline or ground. The 
rotation and pitch of the camera view can be activated while in a travel or stationary 
mode. 

The model is displayed on three Mitsubishi Model VS-5071, 40-inch, rear- 
projection screens set in a semi-circular fashion, sixty-seven inches from the participant, 
providing the user with a 103° field of view (Figure 3.12). Participants are seated behind 


a table that supports the joystick interface. The table has enough room to hold all training 
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materials and doubles as a study station for the participant. This same area is used for in 
briefing all participants and processing paperwork for the experiment. The model display 
and workstation were separated from the lab by four-foot high wall dividers to provide 


privacy and reduce the noise from outside sources. 





Figure 3.12. Model Display Configuration 


The triple screen configuration is used to provide the participant with the 
peripheral cues required for navigation. While maneuvering through a piece of terrain, 
an individual views his position in the context of his surroundings. Without the 
additional terrain features provided on the periphery of an individual’s vision, an 
individual is forced to delineate his position based on the 34.33° field of view rendered by 
a single screen. An example of the additional information provided to the individual is 
seen by comparing the single screen display in Figure 3.13 to the triple screen display of 
Figure 3.14. With the triple screen display, a participant is provided with traits such as 
the lone tree and trail on the left screen and the pavilion and telephone pole on the nght 
screen. Moving through the environment using only one screen, the trees would have 
masked the telephone pole and pavilion to the right. If participants do not stop to look 
around at the intersection or rotate their heads during movement, they will not pick up 
these visual cues. These additional features help individuals to verify that their location 
is the trail intersection 80m to the southeast of Control] Point 8 (Figure 3.15, Item A). 
Without these cues, participants may believe they are at the intersection 75m to the 


southwest of Control Point 8 (Figure 3.15, Item B), the intersection 125m south of 


5) 


Control Point 9 (Figure 3.15, Item C), or the intersection 65m northwest of Control] Point 
8 (Figure 3.15, Item D). 
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Figure 3.15. Map Excerpt for Three Screen Display 
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The effect of a three-screen configuration is even more apparent during 
movement. Figure 3.16 demonstrates how viewable time of a reference point increases 
as the field of view increases. During movement, individuals gauge their speed and 
position on how fast objects enter and leave their field of view. The greater the usable 
field of view the longer an object remains within view. If individuals use a reference 
point as a hand rail to keep them on course, the longer the reference point remains in 
view the more utility it provides. To demonstrate this, the graph in Figure 3.16 displays 
the curves for each field of view plotting velocity vs time. Using a generalized triangle, 
the angle @ and distance a remain constant while the triangles remaining angles and sides 
b and c change as the viewpoint approaches the goal. Beginning movement at a distance 
of 212m from the goal B equals 10°. With a center screen field of view of 30°, the 
reference point will drop off the center screen at a distance of 151m (B equals 15°). 
Expanding the field of view to 100° by using a three-screen configuration, the reference 


point would appear on the side screen from 151m to 58.5m, Bequals 50°. 
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Figure 3.16. Reference Point Visibility Graph 
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Using the above distance, we can calculate the time a reference point is visible for 
each field of view by multiplying the distances by the rate of movement (Table 3.2). At 
lower velocities the difference in usable time is immense. The visible time of the 
reference point becomes more significant at greater speeds. This is due to the time 
required to identify a reference point in the environment and then locate it on the map. If 
the reference point quickly disappears from the center screen, it may not be recognized 
and utilized by the participant. Exposure on the peripheral screens may provide the 


seconds needed by the operator to use these reference points. 


Table 3.2. Visibility Graph Time Table 
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Peripheral cues used during stationary and active utilization of the model provides much 
needed information which will allow participants to disambiguate locations in the 


environment where parallel errors may occur. 


D. PARTICIPANTS 

Fifteen individuals, one female and fourteen males, served as participants in this 
experiment. The group consisted of one civilian and six Army, six Marine, and two Navy 
students from the Defense Language Institute (DLI) and the Naval Postgraduate School 
(NPS). The participants ranged in age from 28 to 39 with a mean age of 32. Participants 
had no prior knowledge of the evaluation area, nor any experience with the VE model. 
Participants were divided into one of three groups, "map only", "real-world", or "VE" 
treatment groups, based on the results of the Guilford-Zimmerman Spatial Orientation 
Aptitude Survey (GZ) (Appendix E.6). Participants received no monetary or academic 


compensation for their participation in the experiment. 
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Data collection occurred over a 71-day period. The extended time frame for data 
collection was to allow the grass to rejuvenate from the traffic created by participants 
moving through the terrain. An additional week’s delay occurred due to an intense 


search of the area by law enforcement and military personnel looking for a missing child. 
E. RESEARCH MONITORS 


My assistant and myself acted as research monitors. Each monitor had gone 
through the course as a participant of MAJ William Banker’s thesis experiment [BANK 
97] or as a pilot participant for this experiment. Researchers followed a specified 
experimental outline (Appendix A) and a series of scripts (Appendix C) to ensure that 
each participant was presented with the same set of instructions, materials, and 
conditions. Whenever possible, participants were observed by both monitors. Research 
monitors carried additional equipment to ensure adequate supplies were on hand for the 


recording of information as well as for the safety of the participants (Appendix J). 
F. TRAIN-UP 


At the beginning of the experiment, participants were briefed on the requirements 
of the study and were asked to sign consent forms (Appendix D). After the initial brief, 
the Self Ability Evaluation (Appendix E.3), the Santa Barbara Sense-of-Direction Scale 
(Appendix E.4), and a map reading test (Appendix E.5) were administered. Next, the GZ 
test was given to measure the participants’ ability to orient themselves in a 3-D 
environment. The result of the GZ test facilitated distribution of participants into the 
three treatment groups. The participants were evenly distributed based on above average 
and below average spatial orientation aptitude scores. 

The first treatment group consisted of map only participants. These participants 
studied a map of the Fort Ord orienteering course. The second, or real world group, was 
given a map of the Fort Ord orienteering course and was allowed to explore the actual 
course. The third, or VE study group, was given the map and access to the real-time | 
virtual environment of the course to study. | 

During the training phase of the experiment, all reasonable attempts were made to 
replicate the procedures of Banker’s 1997 experiment. Each participant was given: a 


Participant’s Task List (Appendix G), Map Marking Instructions (Appendix H), Course 
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Clue Sheet (Figure 3.17), a laminated map of the course (Appendix F), digital photos of 
the control points (Appendix I.2), scratch paper, pencil, and a red alcohol marker to draw 
the planned route. Virtual environment participants were given photos of the actual and 
virtual environment control points (Appendix I.3). Real world participants were also 
given a compass for their study phase. All participants were given one hour to study the 


material provided and plan their routes. 


2070 meters! 11 m [Course Orange Length 2070 meters Climb 11 m 





Figure 3.17. Course Clue Sheet 

During the study phase, a research assistant observed each participant. Participant 
behavior was monitored and recorded on the participant’s training phase worksheet 
(Appendix M.2) for future analysis. Participants were informed when they had 30 
minutes and 10 minutes remaining. 
G. DATA COLLECTION TECHNIQUES 

Upon completion of the study phase, all study materials, except for the clue sheet, 
were collected from the participant. The participant was then taken directly to the 
Starting point of the course by the shortest available route. Participants completed the 
evaluation phase via nine planned tasks, (Appendix B), and three unplanned tasks (Tasks 
3.1, 5.1, and 10). The navigation course was divided into nine unequal legs, requiring 
participants to successfully negotiate nine checkpoints. The total straight-line distance of 
the course was 2070m. 

The participants established planned routes during the train-up phase with the 


participants marking their intended route on the laminated map utilizing a red marker. 
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Participants’ path knowledge was demonstrated by their ability to navigate from each 
succeeding checkpoint along their planned route. Accuracy was measured by calculating 
the number of deviations and the total distance deviated from the participant’s planned 
route. Measurements were taken using a differential global positioning system (DGPS) 


and a Newton MessagePad 130 (Figure 3.18) which were carried by the participant. 





Figure 3.18. DGPS Backpack and MessagePad 130 

The MessagePad 130 registered and stored a coordinate every five seconds as a 
participant moved and every five minutes while a participant was _ stationary. 
FieldWorker software was used to record the information for future analysis. Participants 
were allowed to deviate from their planned routes up to five meters while traveling on 
trails and fifteen meters during cross country movement before an error was assessed. 
This allowed the participants to explore the area and confirm their position without being 
penalized. Participants were also allowed to travel back and forth along their planned 
routes without penalty. 

A helmet camera was used as a second means of data collection. The camera was 
a Hi8 camcorder bolted to the top of a hockey helmet (Figure 3.19). A sighting apparatus 
was fabricated and affixed to the helmet to allow the camera operator to determine if the 
participant was in the field of view. The camera’s focus level was fixed at infinity and 
the camera operator stood at a distance of two or more meters from the participant to 
ensure the best possible image under these irregular conditions. The primary purpose of 
the helmet camera was to record map/compass checks validating the MessagePad’s 


entries, and to provide data for behavioral analysis [BANK 97]. Participants were asked 


4] 


to “think out load” (Appendix K) in order to provide insight to their thought process as 


they moved through the environment. 





Figure 3.19. Helmet Camera 


As a final means of data collection, monitors manually recorded any deviation 
from the participant’s planned route and instances of map/compass checks on a black and 
white copy of the course map. This was done in the event of DGPS failure and was 
another means of verifying map/compass check locations and movement route. The 
monitors’ knowledge of the test area, their experience in creating the VE, and their 
verification of the orienteering map allowed them to reliably record participant route 
deviations and map/compass check locations. 

The experiment also examined survey knowledge by measuring (a) egocentric 
spatiai knowledge using the “wheel” test and an unplanned route selection task, and (b) 
exocentric spatial knowledge using the “whiteboard” test (R. P Darken, personal 
cummunication, October 27, 1997). The wheel test was given to participants at 
checkpoints 2 and 4. The one-hour time limit was suspended during these tests. 
Monitors provided participants with a 12” x 12” plywood platform secured to a four-foot 
long 4x4 post (Figure 3.20). The post was fashioned to fit into a frame anchored in the 
ground near the ~ontrol point. The frame was to ensure all participants are presented with 
the platform in the same location and orientation. On top of the platform, a seven-inch 
color wheel was attached. The wheel contains 16 different colored segments, with three 
pointers fixed to the center labeled SP, 2, and 9 for tasks at CP 2 (Figure 3.21) and 
labeled 1, 6, and 8 at CP 4. Control points were chosen to ensure that only one of the 
controls had been visited by the participant prior to the wheel test and none of the 


controls were either just visited or the control to which the participant was en route. This 
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prevented participants from viewing the controls in context with their planned route, thus 


measuring only route knowledge. 
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Figure 3.20. Wheel Test Platform 

The wheel was affixed to a post, which allowed the participant to move freely 
around it, while the wheel maintained the same orientation for all participants. After 
positioning the wheel table to the south of the checkpoint, the monitor instructed the 
participant to use the wheel and its three pointers to indicate the directions to the 
appropriate control points. The color wheel with no bearing marks was chosen to force 
the participant to rely solely on his survey knowledge of the environment and not to 
confuse the wheel with a compass. 

Monitors recorded actual bearings of each participant’s pointer positions using the 
color-coded segments of the wheel. More accurate measurements were taken in the lab 
using a protractor and digital photos of the participant’s color wheels. Also measured 
was the time it took the participant to complete the task and the participant’s orientation 
while positioning the arrows. Monitors recorded observations and results on the 
participant’s evaluation phase data sheet for future analysis (Appendix M.3). Upon 


completing the wheel test and before continuing on with the planned route, participants 
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were told how much time they had left to complete the course. Additional time checks 


were provided with 30 minutes and 10 minutes remaining. 


Side Towards Checkpoint 





Figure 3.21. Wheel Test 

After completing the planned course, participants were asked to indicate the 
direction of CP 4, describe the best route to CP 4, and finally to navigate to CP 4 using 
the most expedient route without referencing a map or compass (Appendix L.5). 
Participants were evaluated based on the route they traveled, the number of turns that 
lead them away from CP 4, and the distance they deviated from their planned route. 
After reaching CP 4, participants were finished with the navigation portion of the 
experiment. 

Before leaving the course, one final unplanned test was administered. The 
whiteboard test measures a participant’s exocentric survey knowledge of the 
environment. For this test, monitors provided participants with a white magnet board and 
ten magnets corresponding to the start point and the nine respective checkpoints. 
Researchers then instructed participants to place the magnets on the board in proper 
relation to each other, as the points would appear from a top-down view of the terrain 
(Figure 3.22). A digital photo was taken of the participant’s magnet layout and later 


analyzed to measure the accuracy of the representation’s relative bearings between 


checkpoints and the relative distances of each leg of the course. An overall score is 
computed based on the total angular deviation compared to calculations derived from the 


orienteering map (Chapter IV, Section A.5.b). 
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- Figure 3.22. Whiteboard Test 

After completing the whiteboard test, participants were med back to NPS for the 
debriefing. During the debriefing, participants were asked to complete a questionnaire 
about their method of training and the course (Appendix A.7.f). Participants were also 
walked through the route they followed which was plotted on an aerial photograph of the 
course. The data exhibited on the photograph was exported from the MessagePad, 
converted into a format readable by ArcView software, and displayed on a 21” monitor. 
As they were taken through their actual route, participants were questioned on why they 
felt they had deviated from their planned routes. 

During the execution of the course, participants were only required to wear the 
portable GPS backpack and carry their clue sheet. All other supplies, to include drinking 


water, were carried by the research monitors. 
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IV. ANALYSIS 
A. RESULTS 

1. General Information 

The experiment is designed to test a primary and secondary hypothesis 
concerning the navigational knowledge of participants exposed to different training 
methods. To determine overall navigational performance, participants were evaluated on 
their route and survey knowledge of the environment while conducting an orienteering 
course through the target terrain. 

a. Primary Hypothesis: 

Given an hour exposure to training materials, participants with access to a real- 
time virtual environment will outperform those who are exposed to only a map and 
photos of the control points for the same time duration. 

b. Secondary Hypothesis: 

Given an hour exposure to training materials, participants with access to the real 
world will outperform those who are exposed to a real-time virtual environment of the 
same time duration. 

c. Sub Hypotheses: 
1) Route Knowledge 

a) Real world participants will commit fewer errors per control point 
attempted than VE participants. Virtual environment participants 
will commit fewer errors per control point attempted than map 
only participants. 

b) Real world participants will travel less distance per error before 
discovering and correcting their errors than VE participants. 
Virtual environment participants will travel less distance per error 
before discovering and correcting their errors than map only 
participants. 

c) Real world participants will perform fewer map and/or compass 
checks per control point attempted than VE participants. Virtual 
environment participants will perform fewer map and/or compass 


checks per control point attempted than map only participants. 
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2) Survey Knowledge 

a) Real world participants will have a smaller average delta angle on 
the wheel test than VE participants. Virtual environment 
participants will have a smaller average delta angle on the wheel 
test than map only participants. 

b) Real world participants will have a smaller average delta angle on 
the whiteboard test than VE participants. Virtual environment 
participants will have a smaller average delta angle on the 
whiteboard test than map only participants. 

c) Real world participants will have fewer errors during the execution 
of the unplanned route from CP9 to CP4 than VE participants. 
Virtual environment participants will have fewer errors during the 
execution of the unplanned route from CP9 to CP4 than map only 
participants. 

2. Power Analysis 

The tests conducted are two-way analysis of variances (ANOVAs) of study group 
and spatial aptitude (Guilford-Zimmerman). The sample size is fifteen participants. An 
O of 0.05 was used, resulting in a power value (1-0) of 0.1095. As a result, the ability to 
detect alternative hypotheses is poor. This suggests that drawing any conclusions based 
exclusively on a failure to identify a positive effect on any factor is imprudent. A 
multiple analysis of variance (MANOVA) was not performed since each measure was 
analyzed to determine statistical significance for only one sub hypothesis. Simultaneity 
of effects was not considered critical. 

In general, most of the graphs presented here are box plots on primary factors 
indicating the mean, standard deviation, and standard error. In addition, some graphs 
depict extreme data points as dots. 

3. Normalization of Data 

Many of the measurements used for analysis in this experiment occur over time 
and distance. Some participants were not given certain tests because they were unable to 
reach test locations prior to the one-hour time limit. To make participant data 


comparable, several measurements were normalized over the number of controls 
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attempted. This placed each participant’s data in a rational format for correlative 
analysis. 

4. Route Knowledge 

Route knowledge is assessed by analyzing the errors committed, the average 
distance per error, and the average number of checks performed by each participant. 

a. Errors 

Five distinctive navigation errors were observed and recorded during the 
execution of the experiment. These errors were weighted and treated equally in the 
analysis of this data. 

A parallel error occurs when participants mistake one piece of terrain for another 
or travel a parallel path more than 5m off their planned route when traveling along roads 
or trails and 15m off their planned route when traveling cross-country. An example of a 
parallel error is when an individual is located at an intersection west of his perceived 


location (Figure 4.1). 





Figure 4.1. Parallel Error 
Mirror errors occur when participants reach a decision point and mistakenly 
choose to travel the route that takes them in the muror opposite direction of the correct 


path. 


—p - Original Direction of Travel 
— > - Correct Route 
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Figure 4.2. Mirror Error 
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Figure 4.2 is an example of a mirror error would be, if a participant comes to a fork in the 
road and mistakenly chooses the left trail when the right trail was the route originally 
intended. 

Participants are assessed an out-of-bounds error when they travel outside the 
roads that enclose the course. The participants are not informed of their location or the 
proximity of other boundaries. The participants are only told that they have left the 
course area and must return to the other side of the boundary road. No additional 
distance error 1s recorded. 

A reorientation error occurs if participants have been off their designated route 
for fifteen continuous minutes and are not making progress towards their intended control 
point. The participants are stopped, shown their location on the map, reoriented to the 
ground, and given sixty seconds to mark their new route to the intended control point. 
The distance from this location until the participant locates the correct control point is 
added to the parallel or mirror error which was at the origin of the participant’s 
disorientation. 

A compound error is assessed if participants commit a parallel or mirror error on 
the newly planned route resulting from an assessed reorientation error The distance is 
measured and recorded for this error from the location where participants are more than 
5m off their planned route when traveling along roads or trails and 15m off their planned 
route when traveling cross-country until they reacquire the newly planned route. 

Parallel, mirror, and compound errors are weighted equally. Due to information 
being passed to the participant during the assessment of out-of-bounds and reorientation 
errors, they can be thought of as unrequested map checks. These errors are used in the 
calculations of the Map Check Scores (Chapter IV, Section A.4.c) instead of being used 
in determining the participants’ error scores. This prevents the errors from having twice 
the impact on the participants’ overall scores and is more representative of the event. For 
each leg of the course, each participant’s errors were recorded and summed into a Leg 
Error Score. A participant’s Normalized Average Error Score is the summation of the 
individual Leg Error Scores divided by the number of controls attempted (Appendix 
O.3.a). 
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Figure 4.3 shows Normalized Average Error Scores of all three treatments with a 
lower score indicating better performance. The map and real world conditions are 
approximately the same at 0.7 errors per attempt. The VE group lags behind with a group 
mean of 1.1. Also note the real world participants had a far greater variance indicating 
that while some participants had a higher performance others had exceptionally poor 


performance. 
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Figure 4.3. Interaction Box Plot for Error Per Controls Attempted (Group) 
The means between groups are not statistically different, F(2,12) = 1.053, P = .3789. 
Direct observations suggest _that map participants outperformed VE and real world 
participants by committing fewer errors per control attempted. Map participants 
_ appeared to follow their planned routes better indicating they had better route knowledge 
of the course. Real world and VE participants (VE1, VE3, RW3, and RW5) who made 
better progress during the training phase committed fewer errors per control attempt than 
those participants who made little progress during the training phase. This suggests that 
individuals who make it further in the environment during their exposure to a VE or the 
real world demonstrate better route knowledge by committing fewer errors. This also 
suggests that difficulties encountered during the training phase carried over to the 


execution phase of the experiment (Chapter IV, Section B.9). 
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b. Distance Traveled per Error 

Determining how long it takes the participant to recognize and recover from the 
error is as important as identifying when a participant makes an error. To determine this, 
the distance the participant travels is measured from the point an error is committed until 
the participant returns to the planned route. These measurements are determined by 
comparing the participant’s planned route to the data collected from the Global 
Positioning System. Both pieces of information are loaded into and displayed in 
ArcView on the aerial photo of the course. Using the ArcView measuring tool, 
measurements are then taken of any differences based on the route error criteria. The 
experiment parameters specified that measurements be taken from the location where 
participants are 5 or 15m respectively from their planned route until they reacquire their 
planned route was measured and recorded for this error. The measurements are summed 
and divided by the total number of errors to determine the average distance per error. 
These figures are computed for individual legs and the complete course for each 
participant resulting in the Leg Distance Per Error Score and the Distance Per Error Score 
(Appendix O.3.a). Dividing the average distance per error by the number of controls 


attempted results in the Normalized Average Distance Per Error. 
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Figure 4.4. Interaction Box Plot for Normalized Average Distance Per Error (Group) 
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The Normalized Average Distance Per Errors of all three treatments with a lower 
score indicating better performance are shown in Figure 4.4. The map and real world 
conditions are approximately the same at 80m per error per attempt. The VE group lags 
behind with a group mean of 131m. 

The means between groups showed no statistical difference, F(2,12) = .479, 
P = .6305. Direct observations indicate that real world and map participants 
outperformed VE participants by traveling less distance per error per control attempted. 
This implies that map and real world participants were equally adept at identifying errors 
and correcting them. Note that the VE participants had a far greater variance indicating 
that while some participants had a higher performance, one individual had exceptionally 
poor performance. Virtual Environment Participant #2 was an extreme outlier for 
distance measurements since the participant traveled over 3000m on his first error 
encircling the entire course (Appendix N, Figure N.63). After removing this participant 
from the analysis, results for all three groups are roughly identical, (F(2,11) = .003, 
P = .9965). This suggests that there is, in fact, no difference between groups when it 
comes to identifying that an error has been committed and being able to recover from an 


error (Figure 4.5). 
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Figure 4:5. Modified Interaction Box Plot for Normalized Average Distance Per Error 
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c. Map and Compass Checks 

Participants were allowed to request three distinct types of navigational checks 
while on the evaluation portion of the exercise. Each check was timed and recorded by 
research personnel on a black and white copy of the course map. Participants were 
allowed to request as many checks as they felt necessary for them to conduct the course. 
Participants were allowed to request consecutive checks if they need them. During a map 
check, participants were allowed to view the laminated map marked with their designated 
course for thirty seconds. For a compass check, participants were given an orienteering 
compass for thirty seconds. For a map and compass check, participants were given the 
laminated map marked with their designated course and an orienteering compass for sixty 
seconds. When not being used for a map and/or compass check, the research monitor 
maintained all materials. 

Map and compass checks are weighted at 1.0. A combination map and compass 
check is weighted at 1.5 for the additional information that can be gathered utilizing the 
two in tandem. For a change of route, a participant is assessed a .5 weight against a map 
. check. The penalty is assessed due to the information which can be gained while plotting 
the new route on the map, however a full penalty is not assessed since participants must 
have knowledge of their location and the environment if they wish to change their route. 
If participants misidentify their location and therefore plot a bad route from their current 
position, a parallel error is assessed as participants initiate movement from their current 
position. An out-of-bounds error is weighted at 2.0 because participants receive 
additional information from the research monitor who tells participants they have left the 
course boundaries. An out-of-bounds error also indicates that participants have lack of 
knowledge of their location with respect to the course. The reorientation error is 
weighted at 3.0 due to assistance provided by the research monitor who informs 
participants they are off their designated route and shows them exactly where they are on 
the map before the participants plot a new route to the intended control point. 

Map Check Scores were calculated for each leg of the course and a Total Map 
Check Score was calculated by adding the individual leg scores. Dividing the Total Map 
Check Score by the number of controls attempted normalizes the value. This value is 


known as the Normalized Map Check Score (Appendix O.3.a). Figure 4.6 displays the 
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Figure 4.6. Interaction Box Plot for Normalized Map Check Score (Group) 
Normalized Map Check Scores of all three treatments with a lower score indicating better 
performance. The map condition outperformed the other two with a mean of roughly 1.5 
checks per control attempt. The VE and real world conditions were relatively the same at 
approximately 3 checks. The means between groups are not statistically different, 
F(2,12) = .838, P = .4564. Direct observation indicates that the map only group 
participants performed less map and compass checks than VE and real world participants. 
This suggests that map only participants had more confidence in their memory and 
mental maps than the VE and real world participants who required more checks per 
control attempted to resolve differences in their mental maps and the actual environment. 
This is due to map participants concentrating solely on the map during the training phase 
which resulted in a better mental facsimile of the map. 

5. Survey Knowledge 

a. Wheel Test 

Participants were tested twice using the wheel test (Chapter 3.E), once at Control 
Point 2 and again at Control Point 4. Digital photos of the participants’ answers were 
compared to images of the correct answer. Measurements were taken of the angle 
variances. The participant’s absolute values of the angle deviations are summed for each 
control point. Each participant’s total angle deviation for Control Point 2 and Control 


Point 4 wheel test sites are used to determine the participant’s egocentric survey 


aD 


knowledge. The total angular differences are stored in Average Angular Difference CP2 
and in Average Angular Difference CP4 receptively (Appendix O.4.b). Four participants 
were not administered the wheel test at CP4 since they failed to locate the control point. 
To compare scores across participants, Average Wheel Test Angular Variance scores are 
normalized by adding the absolute values of the angular differences and dividing the sum 
by the number of control points the participant identified, 3 or 6 control points. 

The Average Wheel Test Angular Variances of all three conditions with a smaller 
deviation indicating better performance are shown in Figure 4.7. Al three conditions are 
approximately the same at 26.5° per error per attempt. The means between groups 


showed no statistical difference, F(2,12) = .012, P = .9879. Direct observation suggests 
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Figure 4.7. Interaction Box Plot for Average Wheel Test Angular Variance (Group) 
that map and VE participants outperformed the real world group who had a larger 
standard deviation. This deviation indicates a greater variance amongst participants 
within this group with most participants performing well and some participants 
performing poorly. Without this variance, performance by real world participants may 
have shown better results. The two real world participants who had the greatest average 
angle variance (RW1 and RW4), took longer to perform the wheel test and found fewer 
control points than the other real world participants (Appendix O). This indicates they 


had less route and survey knowledge than their fellow real world participants. 
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b. Whiteboard Test 

Each participant’s Whiteboard (Chapter 3.E) results were analyzed to determine a 
participants exocentric survey knowledge. This was accomplished by calculating the 
angle differences between each control point. The angular variances were determined 
using a metric intended to normalize results for simple comparisons across participants. 
The technique begins with capturing results by taking a top down digital photo of each 
participant’s Whiteboard. The image is down loaded to a PC and imported into Adobe 
PhotoShop. Once loaded into PhotoShop, the image imperfections are removed and the 
photo is squared. Using the PhotoShop navigator tool, “x” and “y” coordinates are taken 
for the center of each control point. The coordinates are fed into an Excel spreadsheet 
that calculates distance between successive control points (SP, 1, 2, 3, 4, 5, 6, 7, 8, 9, SP). 
Dividing each leg measurement by the sum total of all distances for each participant 
normalized the distances. The resulting normalized distance measurements are used in 
conjunction with the Pythagorean Theorem to calculate the angles between the successive 
control points. The resulting angles are compared to the actual angles between control 
points. The actual angles are calculated from the “x” and “y” coordinates from a 
digitized course map. The resulting variances and their absolute sums are provided along 
with participant Whiteboard coordinates and measurements in Appendix O.5. Sanitized 


participant Whiteboard images are provided in Appendix N. 
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Figure 4.8. Interaction Box Plot for Average Whiteboard Angular Variance (Group) 
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Figure 4.8 displays the Average Whiteboard Angular Variances of all three 
treatments with a lower score indicating better performance. The means between groups 
are not statistically different, F(2,12) = 1.056, P = .3781. Direct observations indicate that 
on average, map and VE conditions had smaller average delta angle, approximately 21.5°, 
than the real world group, at roughly 27.5°. This indicates that the map and VE 
participants had a better exocentric reference than real world participants. The enhanced 
level of performance by the map group is due to continued exposure to the map. The VE 
participants’ enhanced performance results from their exposure to the top down view 
with a “you are here” arrow. Because of these two aspects of the training phase, neither 
group had to worry. ‘ut becoming lost in the environment. Real world participants had 
to rely on their navig.. .on skills during the training phase to ensure they did not become 
disoriented. The effort required by real world participants to ensure they did not become 
lost reduced the time available to them to study the map and terrain. This resulted in their 
reduced level of excocentric knowledge of the environment. 

c. Unplanned Route Execution 

The unplanned route test was administered after the successful completion of the 
planned navigational task. The task consists of explaining the route the participant would 
take to reach Control Point 4 from Control Point 9 without referencing a map. The 
participant is then required to navigate to Control Point 4 from Control Point 9 without 
the use of a map or compass. Six of the participants completed their planned course and 
were administered the Unplanned Route Task. Of the six participants administered the 
task, only one navigated to Control Point 4 by a means other than the route used to reach 
“ontrol Point 9 from Control Point 8. This was VE Participant Number 1 (Appendix 
J.6). 

None of the participants tested committed an error enroute from Control Point 9 
to Control Point 4. The average time to complete this task was five minutes and twenty- 
eight seconds. The data from the Unplanned Route Task is provided in Appendix O.6. 
For the participants who were administered the test, Rhein performance indicates 
tendencies to travel previously visited terrain in order to link formerly unassociated 
controls. This supports the belief that landmarks are grouped and associated by common 


links even though they have never previously been closely coupled. Since only 40% of 
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the participants were administered this test, no conclusions can be draw between the 
effects of training tools on the performance of this task. 

In conjunction with the Wheel Test and Whiteboard results, these figures indicate 
that the survey knowledge amongst all participants was not affected by the method of 
study. This could be a result of all participants having access to the same map from 
which to gain their survey knowledge. This coincides with Thorndyke’s research [THOR 
82] which showed that survey knowledge is gained through the study of external sources 
such as maps. 

6. Navigational Performance by Training Condition 

The results of the analysis indicate no statistical significance based on training 
condition. In general, direct observations suggest that map participants outperformed VE 
and real world participants. This can be the result of numerous factors including map 
fidelity, spatial ability, and route complexity. 

A direct comparison of results is paradoxical because of the varying route 
complexities. In this experimental paradigm, participant performance on short difficult 
routes was compared to performance on longer easier routes. Problems encountered with 
identifying meaningful performance measurements, recording data, and conducting 
appropriate analysis were magnified since no two routes were identical. Providing each 
participant with a pre-planned route could have alleviated many issues. The result would 
have been an experiment testing the effect of the training conditions on route knowledge. 
Although interesting, a more important issue is how survey knowledge is affected by the 
different conditions. If VEs are to be a valuable navigational tool for the military, they 
must provide survey knowledge of the environment. Survey knowledge of an 
environment allows individuals to vary their routes based on current conditions and 
allows for more efficient movement through the environment. This is why participants 
must be allowed to explore the environment and plan their own routes. 

B. DISCUSSION 

1. Landmark Knowledge 

To determine the participants’ level of landmark knowledge, they were evaluated 
on their ability to locate and identify the control points. While executing the course, 


participants have three possible results on each leg: control point found, control point not 
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found, and time expired in route. Control point found indicates that the participant 
located and touched the appropriate control point. Participants receive full credit for 
locating the appropriate control point. An unfound control point is defined as a control 
point that participants could not locate because they were off their planned route and time 
had expired. A participant receives 33.33% credit towards a control found for attempting 
to locate the control point. If tume expires while a participant is in route to the next 
control point, the participant is given 66.66% credit towards a control found for 
attempting to locate the control point. Participants are awarded this credit if they are 
enroute to the control point and on their preplanned route. The sum of these values is a 
participant’s Landmark Knowledge Score (Appendix O.3). 

Figure 4.9 displays the Landmark Knowledge Scores of all three conditions with a 
higher score indicating better performance. The means between groups are not 
Statistically different, F(2,12) = .563, P = .5840 (Figure 4.1). Direct observation suggests 
that on average, map and real world participants located more controls than VE 


participants. This cursory analysis of the data does not indicate whether the participants 
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Figure 4.9. Interaction Box Plot for Landmark Knowledge Score 
were having the majority of their problems during coarse movement through the 
environment (Appendix Q) or while searching for the control in the general location of 


the objective. A better measurement of landmark knowledge would have been to 


measure the time or distance traversed as a participant searched within the general area of 
the control (25m). 

2. Route Complexity by Training Condition 

Figure 4.10 displays the ISOM Average Planned Route Complexities Scores of all 
three treatments with a lower score indicating an easier route. The VE condition planned 
less aggressive routes than the other two with a mean route complexity of roughly 1.5. 
The map condition followed closely with a mean route complexity of approximately 1.75 
and real world conditions trailed with 2.0. The means between groups are not statistically 


different, F(2,12) = 1.247, P = .3222. Direct observations indicate that on average, 
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Figure 4.10. Interaction Box Plot for ISOM Average Planned Route Complexity (Group) 
participants in the VE group tended to plan less complex routes. This indicates that the 
VE provided participants with information concerning the complexity of the environment 
which they could not gain from the map or failed to gain from the real world. Time 
compressed training allowed VE participants to explore more of the terrain than the real 
world participants were able to traverse during the training phase. This afforded VE 
participants the opportunity to plan less complex routes by taking advantage of the 
information gained from the VE. Map participants did not have the opportunity to 
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translate their propositional knowledge gained of the environment into imagery. This 
may have prevented them from identify more simplistic routes through the course. 
3. Wheel Test Results Visited vs Unvisited Control Points 
a. Visited vs Unvisited Control Points 
A post-hoc analysis was conducted on the average angular differences between 
previously visited and unvisited control points on the Wheel Test. The analysis was done 
to see if participants had better conceptual placement of controls visited vs controls not 


yet visited indicating a better egocentric conception of visited controls. 
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Figure 4.11. Scattergram for Wheel Tests Results by Controls Visited and Unvisited 

A line with a slope of one would indicate similar angular difference between 
controls visited and controls not visited. Horizontal regression lines suggest that 
performance on visited controls was better than those for unvisited controls. Figure 4.11 
exhibits nearly horizontal regression lines for all three groups. This indicates that 
participants had better spatial placement of visited controls than unvisited controls. Map 
participants showed scatted results suggesting that placement of controls in each 
individual’s mental representation of the world varied based on how they grouped the 


controls. Further research is needed to verify these suggested results and to determine if 
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objects further along on the course are more difficult to place within the mental 
representation than objects closer on the route. 
b. Control Point 2 vs Control Point 4 

Additional post-hoc analysis was performed to determine if participant 
performance improved or degraded on the Wheel Test conducted at CP4 compared to the 
test at CP2. The comparison is meant to determine if individual performance improves 
the longer that the participant is exposed to the actual environment. The inference is that 
increased exposure time allows individuals to resolve the differences between their 


mental maps and the actual terrain. 
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Figure 4.12. Scattergram for Wheel Tests Results by Test Location 

Only participants who performed the Wheel Test at both locations were used in 
this comparison. Although no conclusions can be drawn from the results, observations of 
the research personnel suggest that on average, Figure 4.12 shows assorted results 
depending on the group. Map participants had mixed results for average wheel test 
scores at Control Point 2. They showed a clustering of results around the 25° variance 
measurement for their scores at Control Point 4. This produces in a nearly vertical 
regression line indicating improved and standardized results at Control Point 4 with 


assorted results at Control Point 2. 
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The VE group results produce an approximate horizontal regression line. This 
denotes a clustering of results around the 15° mark for their scores at Control Point 2 
while exhibiting more dispersed results At Control Point 4. This suggests diminishing 
performance for the VE participants between the two test sites. Real world participant 
data resulted in a positively sloped regression line that suggests that the average 
performance of individuals remained relatively constant between the two testing sites. 
Overall results suggest that increased exposure to the environment had little effect on 
performance of participants egocentric representation of the environment. 

4. Spatial Ability Post-Hoc Correlation 

To assist in determining additional tests that may predict individual navigational 
performance, individual post-hoc ANOVAs were run with the independent variables 
being the Self Evaluation Bar Test, the Santa Barbara Sense-of-Direction Scale, the Map 
Reading Test, and the GZ Test. The dependent variables were the same as those used for 
analyzing navigational performance (Chapter 4.A.5). No significance was shown when 
running the dependent variables against the independent variables Self Evaluation Bar 
Test, Santa Barbara Sense-of-Direction Scale, and Map Reading Test. Potential 
Statistical significance exists with regards to the independent variable GZ Ability Groups. 
Only the analysis of the GZ Ability Group results will be discussed. All dependent 
variables analyzed use the same criteria utilized in the factorial analysis of the dependent 
variables by the independent variable group (Chapter IV, Sections A.4 & A.5). 

a. Route Knowledge 
1) Errors 

Figure 4.13 displays the Errors Per Control Attempted based on the two Guilford- 
Zimmerman conditions with a lower score indicating better performance. The results do 
not display statistical significance between the two groups, F(1,12) = 4.040, P = .0656. 
Direct observations indicate that on average, participants who scored higher on the 
Guilford-Zimmerman Test committed fewer errors than those who scored lower. This 
suggests that individuals with higher GZ scores are better able to follow their planned 
routes. This could be due to route selection, organization of mental map, or memory 
skills. Further research is needed to determine what factors the GZ score influences and 


what impact these variables have on navigational ability. 
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Figure 4.13. Interaction Box Plot for Errors Per Control] Attempted (GZ) 
2) Distance Traveled Per Error 
The Normalized Average Distance Per Error of the two conditions is shown in 
Figure 4.14 with lower score indicating better performance. The groups show statistical 
significance, F(1,13) = 9.702, P = .0082. Participants who scored higher on the Guilford- 
Zimmerman Test traveled shorter distances per error. This implies that individuals with 
higher GZ scores identify their errors and can recover from them faster than those 


individuals who have lower GZ scores. Since participants with higher GZ scores 
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Figure 4.14. Interaction Box Plot for Normalized Average Distance Per Error (GZ) 
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committed fewer errors and recovered from them faster than participants with lower GZ 
scores, they were also able to attempt more control points. This helped to improve their 
self-confidence as they continued on the course and reduce their scores that were 
normalized by controls attempted. This provides credence to and justification for the 
observation that they were able to obtain better route knowledge of the environment than 
the average participant with a lower GZ score. 
3) Map and Compass Checks 

Figure 4.15 displays the Normalized Map Scores based on the two Guilford- 

Zimmerman conditions with a lower score indicating better performance. The results 


failed to show statistical significance between the two groups, F(1,13) = 4.254, P = .0597. 
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Figure 4.15. Interaction Box Plot for Normalized Map Score (GZ) 
Although no conclusions can be drawn from the results, direct observation suggest that 
on average, participants who scored higher on the Guilford-Zimmerman Test required 
fewer checks and corrections by monitors than those who had lower GZ scores. This 
implies that individuals with higher GZ scores had more confidence in their memory and 
mental maps than individuals with lower GZ scores. This is a result of their spatial 
ability which allowed them to organize a specific set of navigation cues or develop an 
accurate mental map. These skills also allowed them to confirm or modify their mental 
representations as they were presented with conflicting information during execution of 


the course (Appendix Q). 


66 


b. Survey Knowledge 

1) Wheel Test 
The Normalized Average Distance Per Errors of the two conditions with a smaller 
deviation indicating better performance is shown in Figure 4.16. The means between 
groups shows a Statistical difference, F(1,13) = 6.064, P = .0285. Direct observations of 
the research personnel suggest that on average, participants who scored higher on the 
Guilford-Zimmerman Test had smaller average delta angles indicating they had a better 
exocentric reference than lower GZ score participants. This is a result of their enhanced 
ability to fix their position in their mental representations and then rotate their mental 


maps to identify the relative position of the control points based on their location. 
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Figure 4.16. Interaction Box Plot for Average Wheel Test Angular Variance (GZ) 
2) Whiteboard Test 

Figure 4.17 displays the Average Whiteboard Angular Variances based on the two 
Guilford-Zimmerman conditions with a smaller deviation indicating better performance. 
The results failed to show statistical difference between the two _ groups, 
F(1,13) = .128, P = .7258. Direct observations of the research personnel suggest that on 
average, the Guilford-Zimmerman Test results had no impact on the average delta angles 
indicating both higher and lower scoring individuals have similar abilities to acquire valid 


exocentric representations of the environment. Since this test did not require participants 
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Figure 4.17. Interaction Box Plot for Average Whiteboard Angular Variance (GZ) 
to conduct a mental rotation of the environment, spatial ability did not have the same 
affect on performance as it did on the wheel test (Chapter IV, Section B.4.b.1). This 
indicates that excocentric spatial knowledge is independent of egocentric spatial 
knowledge with regards to spatial ability. As a result, survey knowledge must be 
evaluated based on both excocentric and egocentric measures. 
3) Unplanned Route Execution 

The Normalized Average Distance Per Errors of the two conditions 1s shown in 

Figure 4.18 with lower score indicating better performance. The means between groups 


indicates statistical difference, F(1,13) = 7.316, P = .0180. Direct observation suggests 
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Figure 4.18. Interaction Bar Plot for Unplanned Route Execution (GZ) 
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that on average, a participant who scored higher on the Guilford-Zimmerman Test was 
more likely to complete the course in time and be administered the unplanned route task. 
Since participants with higher GZ scores were more likely to complete the course, it is 
more likely they had acquired the survey knowledge required to perform the unplanned 
route task than those individuals who had lower GZ scores. This implies that individuals 
with higher GZ scores have the ability to obtain overall route and survey knowledge 
faster than those individuals who have lower GZ scores. 
c. Route Complexity by Spatial Ability 
1) Route Planning 
Figure 4.14 displays the ISOM Average Planned Route Complexities based.on the 
two Guilford-Zimmerman conditions with a lower score indicating easier route. The 
results suggest statistical significance between the two groups, F(1,13) = 8.614, 


P=.0116. Direct observations indicate that on average, participants who scored higher 
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Figure 4.19. Interaction Box Plot for ISOM Average Planned Route Complexity (GZ) 
on the Guilford-Zimmerman Test planned less difficult routes (1.417, high beginner) than 
those who scored lower (2.422, high intermediate). This implies that those individuals 
with higher GZ scores are better able to identify simple eEnies through this environment. 

The ability of higher GZ score participants to identify and plan more simplistic 
routes is the result of their ability to conduct mental rotations of the map symbols or 


environmental imagery. This allowed them to mentally visualize decision points 
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throughout the environment and determine which features would provide the best 
directional, identification, and reassurance signs (Appendix Q). These signs helped them 
to confirm their position and orientation along their planned route prior to executing the 
course. 
2) Route Planning vs Average Error Score 

Figure 4.20 displays the ISOM Average Planned Route Complexities based on the 
two Guilford-Zimmerman conditions and Normalized Average Distance Per Error with a 
score in the lowe :«ft corner indicating easier route and better performance. The results 
do not indicate statistical significance between the two groups, F(2,14) = 3.710, 


P = .0557. Direct observations indicate that participants who scored higher on the 
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Figure 4.20. Scattergram for ISOM Average Planned Route Complexities (GZ) 


Guilford-Zimmerman Test planned less difficult routes and were able to follow those 
routes better than those who scored lower. This implies that individuals with higher GZ 
scores are better able to plan routes that allow them to identify when they have left their 
routes. This allows them to quickly recover from their errors and continue on the planned 


route. 
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d. Navigational Performance by Spatial Ability 

The results of the analysis of route and survey knowledge based on spatial ability 
suggest possible statistical significance. This implies that spatial aptitude has a more 
profound impact on navigational performance than training effect. This could be due to 
the individuals’ ability to understand the complexity of the task, plan a more appropriate 
route, and resolve differences in their mental maps while executing their planned routes. 
The data does not imply that lower scoring GZ participants could not execute their 
planned route. Instead it suggests that they did not understand the complexity of the task 
and planned routes which were too complex to execute within the allotted time. 

5. Debriefing Questionnaire 

To provide a qualitative analysis of the tools and course used for the experiment, 
participants were given a debriefing questionnaire. The map and real world groups’ 
version of the questionnaire (Appendix E.8) did not have any questions concerning the 
experiment model or its interface. The VE group questionnaire asked specific questions 
regarding the experimental model and interface. A five-point scale (1-5) was used for the 
questionnaire. 

a. Map Questions 

Qualitative analysis of the map indicates that map participants had more 
confidence in the 1:5,000 orienteering map than their real world or VE counterparts 
(Questions MapQ2 — MapQ6). This is partially due to the need for real world and VE 
participants to resolve differences in the mental maps they created during the training 
phase and the course they were running for the execution phase (Chapter IV, Section 
B.11). Real world and VE participants showed less confidence in the map’s ability to 
depict vegetation even though the experiment map used the more descriptive depictions 
of orienteering maps than the traditional military representations. The.mean qualitative 
score was 3.4, adequate representation, and scores ranged from 1 to 5. The participants 
felt the map allowed them to easily plan their routes (Question MapQ7). This indicates 
that the map was a useful tool for all participants. 
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Figure 4.21. Cell Bar Chart for Debriefing Questionnaire (Map Questions) 


Table 4.1. Map Questions 


b. Course Questions 










Qualitative analysis of the course indicates that map participants felt the course 
was moderately difficult and that control points were well marked and located in the 
general location they had expected them (Questions CourseQ1 — CourseQ3). On average, 
participants felt that trails had been trampled down between controls with scores ranged 
from 1 — 4 (Question CourseQ4). This did not help participants locate CP2 or CP4. 
Trails only provided confidence when participants were on their desired route. The trails 
created confusion when participants were disoriented. The reported difficulty of 
remembering one’s planned route ranged from easy to hard with a mean value of 3.667 
indicating that the average participant found this to be moderately difficult (Question 
CourseQ5). Virtual environment participants felt they were better able to remember their 


planned routes than did the map or real world participants. 
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Figure 4.22. Cell Bar Chart for Debriefing Questionnaire (Course Questions) 


How difficult was the course? 
Were the control points well marked? 


Were the control points located where you expected them? 
Had routes been trampled down leading to the control points? 
Did you have difficulties remembering your planned route? 





Table 4.2. Course Questions 
c. Miscellaneous Questions 
Map participants felt they had sufficient time to plan and study their routes 
(Questions MiscellaneousQ2 and MiscellaneousQ3). Virtual environment and real world 
participants felt they could have used more time. Resolving differences in the 
environment requires time and exposure to the terrain (Chapter IV, Section B.11). The 
reasons for this are similar to the rationale behind why participants in the VE and real 
world groups were less confidant in the map (Chapter IV, Section B.4.a). The time used 
to resolve these issues distracts from the time allotted to plan and practice a route through 


the environment. 
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Map participants felt more confident in navigating through the environment 
without the use of the map (Question MiscellaneousQ5). Real world participants felt less 
confident than their map and VE counterparts. This is due to the real world participants’ 
inability to explore the entire environment during the training phase. Four of the five real 
world participants failed to make it past Control Point 6 during the study phase. These 
participants were unsure of the environment on the south side of the course. The 
southern half of the course was depicted as having more undergrowth and greater 
changes in elevation than the northern half of the course (Appendix F.6). This limited 
exposure to the environment translated into a lack of confidence in navigating without the 
use of the map. 

d. Model Questions 

Map participants felt the model correlated well with the map and was easily 
viewable (Questions ModelQ2 and ModelQ1). Participants felt that the elevation 
representation and man-made structures were well represented in the model (Questions 


ModelQ7 and ModelQ4). These elements of the model assisted participants with 
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identifying the general area in which they would locate the controls and enhanced their 


confidence in the training they had received (Questions ModelQ9 and ModelQ11). 
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Figure 4.24. Cell Bar Chart for Debriefing Questionnaire (Model Questions) 



















Were the trails & roads adequately represented in the model? 

ModelQ4 | Were the man-made structures adequately represented in the model? 

ModelQ5 

ModelQ6 

ModelQ7 

ModelQ8 
Q9 

ModelQ10 










ModelQ11 {Do you feel the model gave you an advantage you normally wouldn't have had? 
ModelQ12 |Would you use this tool if it were available for mission planning? 


ModelQ13 |Would you use this tool if it were available for mission rehearsal? 
ModelQ14 |Would you use this tool if it were available for navigation training? 


Table 4.4. Model Questions 
Participants had only moderate confidence in the trail network, obstacles, and 


vegetation representations in the model (Questions ModelQ3, ModelQ5, and ModelQ6). 





This lead to participants having difficulty in identifying the controls once they were 
within 50m of the control (Question ModelQ8) during the execution phase. This is due to 


the low-level vegetation (grass and brush) and smaller depressions of the actual course 
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that were not as detailed in the model. This is more apparent for controls located at 
ground level, Control] Point 2 and Control Point 4, where more errors were made 
attempting to locate these controls than any of the other controls (Chapter IV, Section 
B.7). 

Participants were more likely to use the model for mission rehearsal (Question 
ModelQ13) than for mission planning or training of general navigation skills (Questions 
ModelQ12 and ModelQ13). Participants felt the model provided moderate assistance in 
planning their routes (Question ModelQ11). It is interesting to note that even though 
their overall navigation performance was not as good as the map only participants, the 
VE participants had enough confidence in the model to use it for mission rehearsals. 

e. Interface Questions 
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Figure 4.25. Cell Bar Chart for Debriefing Questionnaire (Interface Questions) 














InterfaceQ7 |Was the 15-minute train-up on the initial model enough time to 
become familiar with the interface? 


Did the use of three screens cause any confusion when maneuvering? 
Table 4.5. Interface Questions 


wall taal Lavell Lavell Ease! 
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As expected from their relative ease in moving through the environment and the 
limited tasks which the participant was asked to perform in the model, VE participants 
felt the model interface was user friendly (Questions InterfaceQ1 — InterfaceQ7). Also, 
the three-screen configuration provided them with no difficulties in viewing the 
environment (Question InterfaceQ8). 

f. Model Needs 

The last page of the questionnaire asked participants to list the items they felt 
would best assist them in navigating through a VE and real world. Streams and rivers 
were deliberately left off the list of possible water features to see if participants would 
pick these as linear features that should be portrayed in a VE. Findings are discussed by 
object groupings: buildings, miscellaneous objects, obstacles, roads, terrain, vegetation, 
and water. 

1) Buildings 
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Figure 4.26. Cell Bar Chart for Model Needs (Buildings) 

Virtual environment participants were more discerning in the types of buildings 
that they wanted displayed in a model. They differentiated between more permanent and 
distinguishable buildings such as factories and public structures (churches, fire stations, 
schools, and government buildings) and the more abundant and ever changing structure 
of houses and shacks. This demonstrates the ability of VE participants to identify the 


more prominent landmarks in the model and disregard the less distinctive objects. 


a 


2) Miscellaneous Objects 
Nearly all the participants sought assistance from directional aids. A compass or 
virtual sun that can provide cues to direction of travel were the most requested items in 
this category. Map participants would also like to have location indicators such as street 
and road signs. Virtual environment participants viewed complex items such as rock 
piles as being useful. This is due to the perception that if a model builder is going to put 


forth the effort to represent such a complex object, it must be a valid landmark. 
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Figure 4.27. Cell Bar Chart for Model Needs (Miscellaneous Objects) 
All participants agreed that a model whose purpose is to provide spatial 
knowledge of an environment does not need to represent movable entities such as people 
ad animals. Sound is also not seen as an essential need for most participants. This is 
to the fact that moving entities and sounds provide little directional cues to the model 
_ or. The exception to this is the sound of a stream or highway noises. If these items can 
be spatially represented in the VE, they can provide navigational cues to the user. On 
weekends when training was being conducted at the Fort Ord Military Operations Urban 
Terrain (MOUT) Site, participants could maintain cardinal directions based on the sounds 
of weapons located southeast of the orienteering course. Sound based navigation cues 
were also provided by motorcycle or Formula One engines when races were held at the 


Laguna Seca Raceway which 1s also located southeast of the orienteering course. 
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3) Obstacles 
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Figure 4.28. Cell Bar Chart for Model Needs (Obstacles) 

Objects that are easily viewable from a distance or present a major impact on 
mobility are highly requested in a VE. Participants requested vertical obstacles and 
elevated obstacles more often then smaller, more easily bypassed obstacles such as pits 
and shallow ditches. This is because the vertical and elevated obstacles are more 
permanent due to the difficulty to construct in the real world and because they can be 
easily seen and used as navigational aids. 

4) Roads 

Participants desire the representation of more permanent man-made linear 
landmarks such as roads. They are less prone to changes and provide a rapid means of 
travel through the environment. Roads also link more prominent landmarks which 
participants use for coarse movement (Appendix Q) through the environment. Trails also 
provide valuable information but, are more subject to change and are therefore less 
reliable. Footpaths change with the seasons and can be easily produced by man or animal 
and are less direct in ther course through the environment. Footpaths may not lead 
participants to their intended destination which leads to a general distrust of paths that 


have not been traveled previously by the participant. 
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Figure 4.29. Cell Bar Chart for Model Needs (Roads) 


5) Terrain 


Terrain elevation plays a major part in determining one’s location in an 


Participants desire adequate representation in changes of elevation. 


environment. 


Terrain which is easily viewable from a distance (hills and ridgelines) was most often 


requested. A contradiction to this is the desire for spurs and fingers to be adequately 


represented but, not their compliment, draws. The two are distinguishable at a distance 


but, a lack of terrain elevation is not seen as important to the participants as the presence 


of elevated terrain. This is likely a result of no draws being present in the testing 


environment. 
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Figure 4.30. Cell Bar Chart for Model Needs (Terrain) 
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One of the major problem locations on the course was Control Point 4 which was 
located in a depression. Because of the difficulty many participants had with this control, 
many of them indicated the need for depressions to be adequately represented in a VE 
used to provide spatial knowledge of an environment. If a control point was located in a 
draw, more participants would have identified this as a type of terrain which must be 
adequately represented in the model. 

6) Vegetation 

Participants identified a lack of vegetation (a clearing) as a vital element of a VE. 
Participants understood that no matter how accurate the placement of vegetation in the 
environment, most of the trees and bushes are randomly placed and therefore should not 
be used as landmarks. Clearings, or the lack of vegetation, is seen as a more defining 
characteristic of a wooded environment, much as an oasis in a desert. It is the differences 
in vegetation which make the area distinguishable from its surroundings and useful in 
determining one’s location. 
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Figure 4.31. Cell Bar Chart for Model Needs (Vegetation) 

Undergrowth and flowers are seen as unnecessary fillers. This is in direct 
contradiction to the responses of the VE participants ae felt the lack of grass and 
undergrowth in the model made it more difficult to locate controls placed at ground level 
(Chapter FV, Section B.4.d). This does indicate that VE participants realize that these 


items are not necessary for general navigation through an environment. Although 
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undergrowth may hinder cross-country movement, it provides little interference with 
visibility at elevations greater than three feet above the model’s surface. 

Since trees and brush do hinder visibility and movement through the environment, 
participants see a need for them to be represented in the environment. This type of 
vegetation provides an indication on which routes provide cover and concealment and 
which routes may make rapid cross-country movement impossible. 

7) Water 

Participants desire the representation of major bodies of water in the VE. The 
reasons are similar to those used for determining the types of roads and trails they would 
like represented (Chapter IV, Section B.4.f.4). Major bodies of water change relatively 


little compared to creeks and puddles that may be here one day and gone the next. 
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Figure 4.32. Cell Bar Chart for Model Needs (Water) 
Streams and rivers were not on the list of water objects provided to the 
participants (Appendix E.8). Two map participants added these to their list of objects 
that should be represented. Streams and rivers can provide the same types of 
navigational cues as paved and dirt roads; however, they are susceptible to course 
changes due to increased precipitation levels and soil erosion. 
8) General Comments 
Map participants showed a greater desire for information than the VE and real 


world participants. They requested more objects be portrayed in the VE than the other 
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two groups. This indicates that VE and real world participants may have a better 
understanding of what landmark and defining terrain features are best used to navigate 
through the virtual world and can be easily identified during movement through the 
actual terrain. 

Although streams, rivers, and bridges were left off the list of potential model 
objects, they should be included in a virtual model. By their nature, they define 
boundaries within the environment and passageways between those boundaries. 
Although dismounted and mounted forces may be able to ford water obstacles, bridges 
still provide important links between sectors and due to their limited number and 
distinctive characteristics, they make excellent landmarks. 

9) Top Six Model Needs 

Objects on the Top Six Model Needs were assigned values based on their 
placement on each participant’s listing. Objects that were designated as the most 
important items were assigned a value of 6, the second most important item as 5, the third 
were assigned a value of 4, the fourth most important as 3, the fifth most important as 2 
and the least most important as 1. After assigning these values for each of the 
participant’s selections, the numbers for each object were added to determine their 
overall value. The objects were then ranked in order from highest to lowest totals and 
displayed on a bar chart (Figure 4.33). 

Terrain elevation is the major focus of most participants. Hills, ridgelines, spurs, 
and fingers rarely have major changes in their shape making them excellent navigation 
cues. The next most requested items were linear objects such as roads, trails, rivers, and 
streams. These objects help to identify boundaries within our environment and provide 
directional cues. Man-made structures were next on the list of items desired by the 
participants. Trees and clearings were included but, not placed high of-the list of needs. 
Other directional tools such as compasses were also noted. The frequency of items 
included on the top six list bears a striking resemblance to the order in which the objects 


were placed on the model developed for this experiment (Chapter II, Section C). 
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Figure 4.33. Cell Bar Chart for Model Needs (Top Six Model Needs) 

Using the items identified as the most essential for use in a virtual environment, a 
stripped down terrain model could be created which portrayed only elevation changes, 
linear features, and landmark models. Vegetation could be represented by color coding 
the terrain skin like a map or placing colored walls indicating the type of vegetation and 
its height. Based on the results of this experiment, one would expect that the symbolic 
nature of the VE would assist participants in identifying prominent landmarks while 
reducing the confusion created by diverse and dense vegetation. This could help to focus 
participants on key features and enhance navigational performance. 

6. Simulator Sickness 

During the experiment, a tendency for simulator sickness showed for the one pilot 
and two VE participants who attempted to run a clean route through the model. These 
participants were all able to make it past Control Point 4 before stating they felt ill. Non 
of the participants made it past Control Point 6 before they had to stop and leave the 
room. Participants were given a time credit if they felt sick and had to step away from 
the model. After a five to ten minute break, the participants returned to the model. None 
of them were able to make it past Control Point 7 before feeling 1] and stopping their use 


of the simulator. 
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The simulator sickness could be due to one or more factors. When the frame rate 
of the model is less than 30Hz [PAUS 92], and the screen refresh rate is less than 70Hz 
for color monitors [BAIL 89}, equilibrium problems may occur. The problems occur as 
individuals identify one rate of motion with their peripheral vision while other neural 
processes perceive a different rate of motion [VAN 90] [EBEN 92]. The critical fusion 
frequency is achieved when the refresh rate has reached a level where a steady image is 
attained; normally this is approximately 60Hz [FOLE 97] [THOM 97]. However, this 
rate can fluctuate plus or minus 20Hz depending on the individual [ROGO 83]. 

The spinning of the billboards and the popping of the forest walls may also have 
played a role. As participants moved through the model, the trees would rotate as they 
passed by the participants’ heads. This may have caused equilibrium problems between 
the middle ear and the optical cues. A participant running a final route also concentrated 
more on the model, taking fewer breaks to look at the map. The constant staring at and 
motion of the model may have played a factor in participants succumbing to simulator 
sickness. 

The fact that none of the participants was able to complete a clean run of the VE 
may have impacted their ability to complete the actual course with limited errors and map 
checks. The best performance of any VE participant was by Virtual Environment 
Participant #1 who reached Control Point 7 before stopping the use of the VE due to 
simulator sickness. 

7. Distinguishing Terrain Elevations 

During the execution of the course, all participants showed some difficulty with 
locating Control Point 4 (Chapter IV, Section B.13). For most participants, this difficulty 
arose as they misread the map and thought the control point was located on a hill instead 
of in a depression. This same problem was apparent in all participant groups. The map 
only participants demonstrated this problem the most. Real World participants 
encountered this problem during the training phase and one participant was never able to 
overcome the error in time to locate the control point. Surprisingly, two of the VE 
participants also displayed this problem, both of whom failed to locate CP4 during the 
evaluation phase. These same participants had difficulty locating the control in the VE. 


The exploration of the environment by the real world participants and virtual 
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environment by the VE participants placed most of the participants in the correct vicinity 
of Contro] Point 4 unless a participant took the wrong trail enroute to the low ground. 

This indicates that identifying the difference between hills and depression on a 
map is difficult for some individuals. With a time-compressed study of the map and the 
environment, many people failed to properly identify depressions. Study of a VE or the 
actual terrain may assist in identifying the improper interpretation of the contour lines 
and helps individuals construct a more accurate three dimensional representation of the 
terrain for their own mental map. 

8. Need for Land Marks to Locate Control Point 

Participants who recognized that Control Point 4 was in a depression and not on a 
hill encountered problems pinpointing the control since it was in a shallow pit surrounded 
by knee high grass and brush. The dense low-level vegetation, positioning of the control 
point below ground level, and the limited landmarks in the vicinity of the contro! point 
made locating the control difficult. The participants who used the lone tree 17.5m to the 
west or the jetty of brush 21m to the south of the control as a landmark had the least 
difficulty locating Control Point 4. Real world Participant #5 was unable to locate CP4 
during the training phase but, realized before he underwent the execution phase that if he 
went to the tree 17.5m west of the control and worked his way back, he would have better 
luck in locating the control. He implemented this strategy during the execution phase and 
walked straight into Control Point 4. 

Control Point 2 was also positioned below ground level. Real world participants 
showed difficulties locating this control point (Chapter IV, Section B.13) although a very 
distinctive landmark, a shed, was located less then 20m to the east. Most participants 
who showed difficulty with this control] point veered to the north of the flag and searched 
the terrain to the north and west. Participants who used the shed as an anchoring point 
and followed the edge of the trees to the south of the shed had little to no difficulty 
locating the control. 

Control Point 7 was positioned below ground level in a trench line. Participants 
who made it this far showed little difficulty locating the control. Most participants 


approached the control from the east and walked straight into the end of the trench line. 
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Other participants intersected the trench somewhere to the west of the control and 
followed the trench to its east end where the control was located. 

The positioning of controls below ground level made them difficult to locate. 
However, the successful use of landmarks in the vicinity of the controls made the 
controls easier to locate. Participants who failed to recognize or utilize easily identifiable 
landmarks found themselves confused and off course. The more distinct the landmark, 
the easier it was for participants to fix their position in the vicinity of the control and 
develop a search plan to locate the control. Submerged controls located at the end of 
linear landmarks or close to very distinct landmarks were easier to locate than submerged 
controls located in areas with limited or indistinct landmarks. 

Effort should be taken to identify and replicate easily identifiable landmarks in the 
VE to assist in locating items or fixing user positions in areas that can be confusing or to 
assist in locating objectives that are well concealed. If not, confusion will occur while 
navigating in the VE which can transfer over to problems with navigating in the actual 
terrain. 

9. Correlation Between Disorientation in Virtual Environment and 

Disorientation in Real World 

A review of training and execution phase routes for the VE and the real world 
participants indicates a possible correlation between the locations individuals where 
disoriented during the training phase and the locations they became disoriented during the 
execution phase. Most participants showed difficulty in maneuvering between Control 
Point 3 and Control Point 4. This was the first leg that required participants to traverse a 
straight-line distance of more than 300m. 

Further research is needed to determine if there is a direct correlation between 
locations individuals become disoriented in the VE and were they become disoriented in 
the real world. If there is a link between the two, VEs can be used to validate mission 
routes and to assist decision-makers in predicting mission success probabilities. Virtual 
environments could also be used to identify trouble spots to bypass, conduct map checks, 


or disambiguate landmarks to ensure individuals do not become lost. 
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10. Banker Participants vs Goerger Participants 

a. Differences in Models 

The model used in MAJ Banker’s experiment [BANK 97] was a non-real 
time representation of the environment developed using a golf course creator tool. The 
tool allowed for a very detailed model that had many characteristics of a map. The 
surface of the model was colored in a fashion that produced clean edges between the 
different types of vegetation. This gave the surface a map equivalent characteristic and 
provided the users with the ability to easily distinguish the difference between forested 
areas and clearings. The differing grass colors produced an effect similar to moving from 
covered terrain to open terrain as it delineated region changes. Since the model was 
developed using a golf course tool, users had to select which portion of the course they 
wished to explore and teleport between holes in order to view different portions of the 
model. This was not a seamless transition, as it required participants to refer to a master 
layout to determine which golf course hole and orientation they deeded displayed. 

The model used in this experiment was a real time representation of the 
environment in which users could seamlessly traverse the entire course area. The 
relatively uniform color of the model surface covered with an aerial photograph helped to 
render shadows but, provided no sensation of having an overhead canopy. Appendix P.1 
outlines some of the other differences between the two models, the real world, and the 
map Only training conditions. 

b. Similarities in Performance 

To make comparisons between the two experiments, participants’ experience 
levels for this experiment were reclassified (Appendix O.2) in accordance with 
MAJ Banker’s participant experience level criteria [BANK 97]. Although performance 
levels were lower in this experiment compared to the Banker experiment, relative 
comparisons can be made between training conditions and ability groups for the two 
experiments. Similarities are shown for Total Error Distance by Ability Group (Figure 
4.34), Distance Per Error by Ability Group (Figure 4.35), and Map Checks by Ability 
Group (Figure 4.36). This indicates that MAJ Banker was correct in his conclusions that 
ability group has a limited impact on a participants ability to recognize and recover from 


an error. 
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Figure 4.34 displays Total Error Distances based on Banker Ability Groups for 
participants in this experiment with a lower score indicating better performance. The 
results do not indicate statistical significance between the two groups, F(1,13) = 1.702, 
P = .2147. Direct observation suggests that participants in the Beginner Ability Group 
traveled further off their planned route than the Intermediate participants who were better 
able to maintain their planned course. This fact is relevant only when viewed in context 
with errors committed or total route distance. A participant who traveled 300m off their 
planned route and crossed over 3000m of the planned course, performed better than a 


participant who traveled 2000m, 250m of which was off their planned route. 
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Figure 4.34. Interaction Box Plot for Total Error Distance (Banker Ability Group) 

Figure 4.35 displays Distance Per Error based on Banker Ability Groups for 
participants in this experiment with a lower score indicating better performance. The 
results do not indicate statistical significance between the two groups, F(1,13) = 1.847, 
P = .1973. Direct observation suggests that participants in the Beginner Ability Group 
traveled further per error committed than Intermediate participants who were better able 
to identify when they had deviated from their planned route. Beginner participants had a 
more difficult time recognizing their errors, fixing their position and orientation in the 
environment, and developing strategies to recover from their errors. This is to be 


expected since their navigation skills were limited in comparison to the Intermediate 
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navigators who had a better understanding of the task and a higher level of confidence in 


their skulls. 
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Figure 4.35. Interaction Box Plot for Distance Per Error (Banker Ability Group) 
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Figur 4.36. Interaction Box Plot for Map Checks (Banker Ability Group) 

The Map Checks of the two conditions is shown in Figure 4.36 with a lower score 
indicating better performance. The means between groups does not indicate a statistical 
difference, F(1,13) = .447, P = .5154. Direct observation suggests that participants in the 
Intermediate Ability Group had a greater variance in the number of map checks 


performed per participant than Beginners. This, in conjunction with the number of errors 
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committed, indicates that many intermediate participants performed maintenance map 
checks to ensure they were still on their planned route. 
c. Differences in Performance 

Although MAJ Banker’s thesis did not discuss the following measures, there 1s a 
difference in performance between participants in his study and participants in this study 
for Controls Attempt (Figure 4.37), Controls Found (Figure 4.38), Errors Per Control 
Attempted (Figure 4.39), and Distance by Training Condition (Chapter IV, Section 
A.4.b). Measurements for MAJ Banker’s participants (Appendix P.2) indicate a more 
level performance across ability group, where this study indicates better performance by 
individuals rated as intermediates over those rated as beginners by the criteria outlined in 


MAJ Banker’s experiment. Figure 4.37 displays the Controls Attempted based on Banker 
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Figure 4.37. Interaction Box Plot for Contro] Attempt (Banker Ability Group) 
Ability Groups for participants in this experiment with a higher score indicating better 
performance. The results suggest a statistical significance between the two groups, 
F(1,13) = 5.226, P= .0295. The graph implies that participants in the Intimidate 
Ability Group are more likely to attempt a control than participants who are classified as 
Beginners. 

Figure 4.38 displays the Controls Found based on Banker Ability Groups for 
participants in this experiment with a higher score indicating better performance. The 


results indicate statistical significance between the two groups, F(1,13) = 5.987, 


9] 


P = .0295. The graph suggest that participants in the Beginner Ability Group are less 


likely to find as many controls as participants who are classified as Intermediates. 
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Figure 4.38. Interaction Box Plot for Controls Found (Banker Ability Group) 


Unlike the Banker study, when comparing controls found by treatment group and 


Banker Ability Group, the Intermediate VE participants did not locate more controls than 


their real world and map participant counterparts (Figure 4.39). The plot indicates no 
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Interaction Bar Plot for Controls Found 
Effect: Group * Banker Ability Level 
Error Bars: + 1 Standard Deviation(s) 
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Figure 4.39. Interaction Bar Plot for Controls Found 
(Treatment and Banker Ability Group) 
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significance between training conditions based on Banker’s Ability Levels. The 
difference in performance between Banker’s results and this experiment’s findings is due 
to the more simplistic nature of the Banker model. Since participants could not move 
quickly through Banker’s virtual environment, they were forced to focus their efforts 
when using the model (Chapter IV, Section B.10.d). In other words, for the Banker 
study, the VE and map groups were nearly identical. If intermediate VE participants in 
this study would have focused their efforts around control points and major decision 
points utilizing the teleport option and top down view more, their ability to find control 
points would have been more comparable to Banker’s participants. 

The Errors Per Controls Attempted of the two conditions is shown in Figure 4.40 
with lower score indicating better performance. The means between groups does not 
indicate a statistical difference, F(1,13) = 3.757, P = .0746. Direct observation suggests 
that participants in the Intermediate Ability Group make fewer errors per control attempts 


which implies they are better able to stay on their planned routes than Beginners. 
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Figure 4.40. Interaction Box Plot for Errors Per Controls Attempted (Banker Ability Group) 
Similar to the comparison of controls found by treatment group and Banker 
Ability Groups, the Intermediate VE participants did not attempt more controls than their 
real world and map participant counterparts (Figure 4.41). The plot suggests no 
significance between training conditions based on Banker Ability Levels. Once again, 


the difference in performance between the Banker’s findings and this experiment’s 
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results is due to the more simplistic nature of the Banker model (Chapter IV, 


Section B.10.d). 
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Figure 4.41. Interaction Bar Plot for Controls Attempted 
(Treatment and Banke: Ability Group) 


An identical 2-way ANOVA was run comparing Map Check Score and Total 


Error Distance by treatment group and Banker Ability Groups. Unlike the Banker 
experiment, no statistical significance was shown between training conditions based on 
Banker Ability Levels for Map Check Scores, F(2,9) = 1.602, P = .2539. Nor was there 
any statistical significance shown between training conditions based on Banker Ability 
Levels for Total Error Distance, F(2,9) = 1.522, P = .2695. The Banker study found that 
his VE 4d a Statistically significant increase in performance in these two areas for 
intermediate participants. Banker’s findings were not supported by this study. This is 
due to differences in the type of navigation experience between the two participant pools. 
d. Reasons for Performance Differences 

As a whole, Banker’s participants performed better than the participants of this 
experiment. This is due to differences in the participant pools and the structure of the 
experimental design. MAJ Banker’s participants were more experienced in sports 
orienteering and the use of orienteering maps than participants of this experiment. They 


were also more familiar with running orienteering courses in typical central California 
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coastal terrain since many of them had participated in events in the San Francisco Bay 
area, Santa Cruz Mountains, and Monterey Peninsula. This resulted in a difference in the 
type of experience each intermediate group possessed and the kind of information they 
were able to extract from the VE, map, and natural environment. Due to the inexperience 
of many of the participants in this experiment, their planned routes were more difficult 
than those in the Banker experiment. [If this experiment’s participants had more 
experience with navigating in central California coastal lands, they would have been able 
to extract more pertinent information from the study materials and improved their overall 
navigational performance. This indicates a possible use of virtual environments to train 
navigation skills for areas which soldiers may not routinely encounter. If generic 
navigation skills can be taught through the use of generalized terrain models for desert, 
arctic, mountainous, jungle, and wooded terrain, the VE would provide a useful tool for a 
commander’s training program. 

The experimental outline of MAJ Banker’s thesis was less intrusive to the 
participants. All pointing tasks were conducted prior to the execution of the planned 
route. Once the planned route was initiated, no planned interruptions were made. For 
this experiment, pointing tasks were interjected at Control Points 2 and 4. This 
interrupted the flow of the planned route and may have had an effect on participant recall 
of their planned route. However, most participants showed little difficulty in navigating 
to the control points immediately following the Wheel Tests. Participants also received 
assistance from monitors if they were off course for more than 15 continuous minutes and 
were not making progress towards their designated control (Chapter IV, Section A.4.a). 

The poorer performance of this experiment’s VE participants is due to their 
reduced ability levels and the ability of MAJ Banker’s Non-Real Time VE to provide an 
exocentric as well as an egocentric view simultaneously. This reduced the effort required 
by participants to locate themselves in the environment since the computer model 
resolved this issue for them. Since the model was a non-real time model, participants 
teleported between control points and decision points in the environment instead of 
conducting cross-country movement through the model. Participants in the real time 
model had to navigate through the VE and determine their location as they moved. This 


required them to utilize the navigation cycle (Appendix Q) during the study phase and 
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use training time to move through terrain which provided few disambiguating features to 
assist them with navigating through the actual terrain. If participants had used the real 
tume VE to explore the area around the control points and to identify the differences 
between key decision points along their planned route, their performance would have 
improved. 

11. Map Resolution 

The map used for this experiment far exceeds the capabilities of most maps used 
during traditional military operations or orienteering competitions for its level of detail. 
Any dismounted infantry or special operations soldier would treasure a map of such 
detail when entering into a new area of operations. Most military operations maps are at 
a scale of 1:50,000 or 1:24,000 (Appendix F). 

Because of the small scale of the map and the use of orienteering terrain 
classification markings, participants could glean information from the map that would 
normally not be available to them. Most participants gave the map above average ratings 
for clarity, information provided, and ease of use (Chapter IV, Section B.4.a). The 
increased detail provided participants with enough information that they did not need to 
use the VE or real world to discover and catalog many landmarks or changes in terrain 
elevation that would normally be too indistinct to appear on most maps. This resulted in 
map group participants performing much better than would normally be expected of an 
individual who was provided with only a map, objective photos, and objective locations. 

12. Resolving Ambiguities In Mental Maps 

When participants began the course, they first had to identify their position and 
orientation (Appendix Q). Once this was accomplished, they checked their mental maps 
and list of instruction to determine their course of action. Once movement began, they 
were continuously cycling through a series of mental processes to verify their position, 
orientation, mental map, and route. While updating their mental maps, each group was 
faced with a unique set of issues. | 

Map participants tended to navigate using propositional knowledge, a list of 
directions, that when linked together would lead them through the course. This 
represents’ exocentric knowledge of the environment that is spatial but, not temporal 


which meant that the fidelity of the environment did not encumber map participants. 
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Often they concentrated on more definable characteristics of the course such as buildings 
and trail intersections as well as the distance between them, rather than less 
distinguishable objects such as trees. This required the translation of propositional 
knowledge into ambiguous Static imagery (Appendix Q). Since their imagery was 
indistinct, map participants did not panic when their mental maps did not match the 
actual imagery they encountered. Participants did have to resolve the differences 
between perceived distances on the map and actual distances on the ground as well as 
visualizing the different categories of terrain depicted on the map. Participants who paid 
close attention to the first couple of trails they encountered during the initial portion of 
the course, quickly resolved this issue and showed little difficulty with perceived 
distances for the rest of the experiment. 

Real world participants were faced with a different set of issues as they attempted 
to resolve differences in their perceptions of the real world and the actual terrain they 
were standing on. Similar to the map participants, those real world participants who paid 
close attention to the first couple of trails they encountered during the initial portion of 
the course, quickly resolved the distance issues and showed little difficulty with 
perceived distances for the rest of the experiment. Those real world participants who 
failed to resolve this issue during the training phase showed difficulties with judging 
distances during the middle portion of the course. 

Although real world participants had traveled the terrain once before, their 
perception of the environment was based on dynamic imagery (Appendix Q). Dynamic 
imagery is similar to a mental movie. The navigational performance of real world 
participants is based largely on how well they developed their mental movie during their 
initial run through the environment. If participants make it only partially through the 
environment or become confused along their route, their movie becomes a poorly edited 
collection of three-dimensional clips. The movie remains that way until they can make a 
clean run through the course editing their movie, clarifying discrepancies as they move 
through the environment. Real World Participant #1 demonstrated difficulties in editing 
his mental movie due to the inordinate amount of time he spent looking for Control Point 
3 during the training phase. He searched the area between Control Point 2 to the western 


boundary looking for Control Point 3. Because of this, his mental map and dynamic 
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imagery of the environment was cluttered with ambiguous representations. As a result, 
he was unable to locate Control Point 2 without crisscrossing the area between Control 
Point 1 and Control Point 3 four times. Participants with well edited mental movies 
performed movement along their planned route better than participants who had poorly 
edited depictions of the environment. During the training phase, real world participants 
showed the same difficulty with resolving distances that the map participants initially 
encountered during their execution phase as they dealt with resolving differences in 
propositional knowledge with the actual terrain. 

Virtual environment participants had to deal with many of the same issues which 
the real world participants faced in resolving inconsistencies with their dynamic imagery 
of the environment and their perceptions of distance. Misperceptions of distances and 
sizes from use of the VE caused many VE participants to over estimate the distances they 
need to travel when operating on the actual terrain. The placement of a HMMWV at the 
Start point of the VE did not appear to alleviate these perception issues. Virtual 
Environment Participants #1, #2, #3, and #5 all commented on their initial difficulty with 
resolving distance during the execution phase of the experiment. Similar to the map and 
real world participants, those VE participants who paid close attention to the first couple 
of trails they encountered during the initial portion of the course quickly resolved the 
distance issues and showed little difficulty with perceived distances for the rest of the 
experiment. 

To compound the issues of perception, VE participants had to resolve differences 
between the model environment and the actual terrain. Differences in vegetation density 
and complexity as well as perspective issues required additional mental manipulations in 
the environment. Participants who interpreted the vegetation in the VE as symbolic 
representations of vegetation did not regard vegetation as a landmark. object and were 
able to disregard the vegetation while moving through the actual environment. This 
allowed them to focus on more distinguishable landmarks. 

Participants are willing to accept oversights with maps and VEs, but, are 
unwilling to accept errors (Figure 4.42). If the map or VE is missing something that is 
present in the real world, individuals accept this as changes in the environment since the 


model or map was developed. In other words, the map and VE are assumed to be a 
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subset of the real environment. They resolve the differences with their mental map by 
adding the missing feature. If something appears on the map or in the VE, individuals 
expect to see that same object or feature in the real world. When individuals do not see 
the feature in the real world, confusion may result and they question the validity of the 
map or VE. This can result in total mistrust of the mental map created from a physical 


map or VE. 


Real World Real World 


Abstraction Abstraction 


Acceptable Unacceptable 





Figure 4.42. Venn Diagram of Real World Abstractions 

It is better to leave something off of a map or out of a VE than to misrepresent it. 
Misrepresentation of objects or features on maps and in VEs creates confusion and 
mistrust which leads to individuals abandoning their mental maps and questioning their 
knowledge of the environment. Ultimately this results in a large drop in navigational 
performance. Exposure to maps still provides ambiguity issues. Maps are excellent at 
providing propositional knowledge that produces superior static imagery for route 
knowledge with one caveat. Static imagery is easily fooled by parallel errors (Chapter 
IV, Section A.4). Map Participant #3 felt his biggest problem was resolving distances. 
This lead to his parallel errors at Control Point 2 when he was searching the clearings to 
the north of the building instead of to the west. With a more dynamic image of the area, 
he would have been able to recognize the difference in the clearings such as the open area 
to the south of the clearing in which CP2 was actually located. Map Participant #4 
experienced parallel errors in locating Control Point #3 since from the map study, he was 


unable to distinguish the differences between the buildings located in the vicinity of CP3. 
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The exposure to the actual environment or an accurate VE can help to resolve 
these ambiguities by filling in the gaps by changing static imagery into dynamic imagery 
which can clarify discrepancies that lead to parallel errors. Virtual Environment 
Participant #1 commented that he wished he had used the model to study the area more 
carefully around Control Point 4 and Control Point 9. This would have helped him 
disambiguate the terrain at these locations. Virtual Environment Participant #2 gained 
tremendous confidence in his location when he saw the building near Control Point 2. As 
he approached the area from the north, he stated, “I know from the model that I have to 
go left at this broken down building to find CP1.” During the debriefing, he commented 
that he remembered the pavilion near Control Point 3 from the model which helped him 
fix his position when searching for the control. 

13. Map Checks vs Errors 

MAJ Banker briefly discussed the correlation between map and compass checks 
and distance off route [BANK 97]. His results indicated that the further a participant was 
off the planned route the more likely the participant was to conduct a check. A similar 
simple regression was performed in this experiment to see if the same findings would 
hold true. For this analysis, the Normalized Average Distance Per Error (Chapter IV, 
Section A.4.b) and the Normalized Map Check Score (Chapter IV, Section A.4.c) for 
each participant was used. These two measurements were used because the 
measurements are normalized to take into account the number of errors committed and 
the number of control attempted. 

The results were comparable to the Banker study. The results show a direct 
correlation between the distance participants were off their planned route and the number 
of checks they performed, F(1,13) = 32.380, P = .0001 (Figure 4.43). The results indicate 
that participants who are on their planned route are more confident in their performance 
and conduct fewer checks. Once participants recognize they are off their planned route, 
they conduct checks in accordance with the distance they have deviated from their 
planned route. The further participants were off their planned route, the more checks it 


took them to return to the proper route. 
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Figure 4.43. Regression Plot for Total Error Score Per Control Attempted vs Normalized 
Map Check Score 

14. Average Distance Off Route 

In MAJ Banker’s thesis, he briefly reviews the distance participants veered off 
their planned route per control point [BANK 97]. His conclusions indicated that the real 
world participants’ performance declined as they traveled further through the course 
because they were unable to explore their entire routes during the training phase due to 
the one-hour time constraint. Similar conclusions were drawn from this experiment 
based on the fact that only two of the real world participants were able to make it further 
through the course on the execution phase than on the training phase. This issue can be 
addressed by time compressed training in a VE that allows individuals to more rapidly 
explore an environment through increased speed of movement or teleportation. 

Making comparisons to MAJ Banker’s scatter plot of Treatment Group Distances 
Off Route by Control] [BANK 97] and Figure 4.44, we see an interesting correlation 
between the average distance per participant on Control Point 4. The notably marked 
increase in average distance per participant indicates that Control Point 4 presented 
participants of both studies with a more difficult task. The reasons for this were 


discussed previously (Chapter IV, Sections B.6 and B.7). 
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Figure 4.44. Scatter Plot of Average Distance off Route Per Control Attempted 
Although Figure 4.44 indicates VE participants having even more difficulty with 
Control] Point 4 than the real world and map only participants, Figure 4.45 indicates that 
the difficulty with the control is actually more uniform across the groups. This scatter 


plot displays a normalized distance per error. 
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Figure 4.45. Scatter Plot of Average Distance Per Error Per Control Attempted 
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Both graphs indicate a more level performance after Control Point 4. However, 
only 11 participants made it past Control Point 4 with only 66% of the participants 
making it past Control Point 6, and only the better navigators making it through the 
course. The plot shows that once an error is committed, the mean performance for 
participants demonstrates the same amount of difficulty in recovering from the error for 
Control Point 4 regardless of training condition. 

Figure 4.45 also indicates that map and VE participants had more difficulty 
recovering from errors during movement from the starting point to the first control. This 
is due to both groups of participants attempting to resolve resolution differences between 
their mental maps and the real world (Chapter IV, Section B.11). Virtual environment 
participants showed a marked improvement in their ability to recover from their errors 
from Control Point 1 to Control Point 3, but, ran into difficulty with Control Point 4. 
After Control Point 4, performance for all remaining participants leveled off except for a 
decrease in performance at Control Point 7 for the map participants. The real world 
participants demonstrated the most difficulty of all participants who executed Control 
Points 5 through 9. This supports MAJ Banker’s conclusions that real world participants 
have more difficulty with the later stages of the course because of their inability to 


traverse this section of the terrain during the training phase. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 


1. General Conclusions 


This experiment studied the effects of training methods on spatial knowledge of a 


natural environment given a one-hour exposure to a high resolution 1:5,000 orienteering 


map, access to the orienteering course, or a high fidelity real time virtual representation. 


The following conclusions are drawn from both the quantitative and qualitative results: 


a. Experiment training conditions show no significant effect on an 
individual’s ability to obtain and demonstrate spatial knowledge of a natural 
environment (Chapter IV, Section A.6). 

b. Spatial ability plays a significant role in an individual’s ability to 
obtain and demonstrate spatial knowledge of a natural environment (Chapter IV, 
Section B.4). 

c. Exposure to the actual terrain or a virtual representation of the terrain 
eliminates ambiguities in an individual’s mental map by providing dynamic 
imagery to disambiguate propositional knowledge gained from maps (Chapter IV, 
Section B.12). However, there are other issues with walking the ground or VE 
that prevent these training tools from being better than a really good map for short 
exposure durations. 

d. A high resolution 1:5,000 orienteering map provides an inordinate 
amount of detail which is uncommon in typical maps used for military operations 
and has a significant effect on an individual’s ability to obtain and demonstrate 
spatial knowledge of a natural environment (Chapter IV, Section B.11). It’s hard 
to beat a really good map. 

2. Performance by Study Group 
Based on the use of a 1:5,000 orienteering map and a high fidelity real time VE, 


the results suggest that provided with an hour to study the environment and plan a route, 


map only participants gained more spatial knowledge of a 1km square piece of terrain 


than VE participants. VE participants performed on par with real world participants. 


Overall comparisons of results indicate that map participants outperformed real world 


participants who outperformed VE participants in route knowledge. Results suggest that 
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VE participants had slightly better survey knowledge than map and real world 
participants. 

3. Performance by Spatial Ability 

Using a high resolution 1:5,000 orienteering map and a high fidelity real time VE, 
results suggest that provided with an hour to study the environment and plan a route, 
participants with above average spatial ability scores gain more spatial knowledge of a 
l1km square piece of terrain than participants with below average spatial ability scores. 
Overall comparisons of results indicate that participants with above average spatial 
ability scores outperformed those with below average spatial ability in route knowledge. 
Results suggest that above average spatial ability participants had better egocentric 
knowledge of the environment and slightly better excocentric knowledge than below 
average spatial ability participants. 

4. Mental Maps 

Behavioral analysis in this experiment suggests that the earlier models of 
navigation based on the assumption that individuals, while navigating, make decisions 
depending on a comparison between what they see versus what thy expect to see 
[PASS 84] [JUL 97], are only partially correct. Virtual environments and the real world 
provide us with mental imagery of our environment. Maps only provide us with 
propositional knowledge that most people translate into an ambiguous series of static 
mental images as they move through the environment. Exposure to VEs or the real world 
assists individuals in filling the gaps, resulting in dynamic mental imagery, 
disambiguating images created by the study of a map. With enough exposure to the real 
world and VEs, we can reso'!ve any discrepancies in our mental imagery which enhances 
our confidence in our navigation through the environment. An individual’s preconceived 
egocentric view, whether developed from the study of the real world or VE, provides a 
strong mental image. A participant cannot help but refer to this mental image, even if it 
is incomplete or confused. 

5. Map Resolution 

The performance of the map only subjects may have been skewed based on the 
scale and resolution of the map used for the experiment. This map (1:5,000 orienteering 


map), provided more information than is usually available on maps used for most military 


operations. Providing participants with a map of less resolution will surely produce 
worse results for the map only group and improve the perceived performance of the VE 
and real world groups. 
B. SIGNIFICANCE 

1. Study Method 

Given a limited timeframe and highly detailed maps, individuals will gain more 
information about the target environment than subjects afforded high fidelity real time 
models. Given less than an hour time to prepare for a mission, individuals should 
concentrate more on map study and route memorization than general terrain 
familiarization if they wish to maximize their performance. This fact helps to limit 
expenditure of resources and reduces the possible confusion of mission forces, allowing 
them to focus on those assets which will best assist them in preparing for the operation. 

2. Spatial Ability 

Identifying individuals with above average spatial ability will assist in predicting 
which personnel may be better suited as navigators. If these visual and organizational 
traits can be identified and taught, it will assist in training individuals to perform 
navigation tasks in a manner which will improve their spatial knowledge and overall 
performance in the area of operations. 

3. Mental Imagery 

Giving a relatively unconstrained timeline for mission preparation, individuals 
can make numerous runs through a VE. This will allow individuals to edit their mental 
movies providing them with an excellent three-dimensional mental representation of the 
environment to assist them during navigation. 

4. Map Resolution 

Maps with high resolution provide a tremendous amount of information. If 
individuals have a limited amount of time to prepare for a mission, a high-resolution map 
is a better tool for gaining spatial knowledge of an area. The same detail required to 
produce a high fidelity VE is the same information required to produce a high resolution 
map which depicts vegetation densities and building orientations. Producing a high- 
resolution: map is less time intensive and results in a two dimensional representation 


which is easily carried and studied. With a limited amount of time for mission 
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preparation, resources should be placed on producing a high-resolution map that can be 
quickly mass reproduced and distributed to the mission force. This will save precious 
resources while affording individuals with the best training tools for the available time. 
C. CONTRIBUTIONS 

The experiment implemented two tests that were. developed to help test an 
individual’s excocentric and egocentric knowledge of the environment. These tests were 
essential in determining if individuals had gained only route knowledge of the 
environment or if they were able to obtain and demonstrate survey knowledge. 

1. Wheel Test 

To measure egocentric knowledge of the environment, the Wheel Test was used. 
This test is a variation of the one initially developed for use in an experiment conducted 
to test the transfer of spatial knowledge from a VE to a complex man-made structure 
[GOER 98]. The test measures an individual’s ability to identify the direction to several 
locations within the environment without providing the individual with directional cues 
(Chapter III, Section G). It requires that individuals understand their relative position 
within the environment with regards to locations they have or will be visiting while 
navigating through the environment. Variances from actual measurements are calculated 
and combined for comparison with other participants. 

2. Whiteboard Test 

To measure exocentric knowledge of the environment, the Whiteboard Test was 
used. This test is also a variation of the one initially developed for use in an experiment 
conducted to test the transfer of spatial knowledge from a VE to a complex man-made 
structure [GOER 98]. The test measures an individual’s ability to identify the relative 
position of the control points to one another without any external reference cues (Chapter 
III, Section G). It requires that individuals understand the relative position of the controls 
to each other without worrying about a distance scale. Angles between each control point 
are measured and used for comparison with the actual measurements and to compare 


against other participant results. 
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D. FUTURE WORK 

1. Displays 

During the experiment, participants were exposed to a large three-screen display 
with over 103° field of view and 4800 square inches of viewable surface. This is done to 
provide the participant with the largest viewable surface possible. Little research has 
been conducted to determine which type of display provides the best possible 
environment for navigating. Single screen, triple screen, head mounted display (HMD), 
and wide screen views are just a few options available which are easily configured for 
use at most simulation centers. 

Usability participants in this experiment and in Sullivan’s helicopter navigation 
experiment [SULL 98] indicated that the wider field of view dramatically assisted in 
navigating through the environment. Participants felt they were able to extract 
navigational cues from the terrain on their peripheries much as they would do in the real 
world. This suggests that a wide screen or three-screen configuration may be the best 
display for such training and mission preparation devices. However, these participants 
were not exposed to either environment using a HMD. 

Some users appear to be more prone to simulator sickness than others. This can 
be due to many factors from direct exposure time to refresh rate (Chapter IV, Section 
B.6). During this experiment, participants who made it through the environment and 
were conducting a final run, expressed a feeling of simulator sickness between Control 
Point 4 and Control Point 5. Each of these participants stopped their movement and took 
a five to ten minute break before going back into the simulation. None of these 
participants completed their second trip through the model as they all stopped somewhere 
between Control Point 6 and Control Point 7 even though they had time remaining. 

Further research is needed to determine which display and frame rate are best to 
reduce the instances of simulator sickness while providing the user with the best possible 
display for obtaining the information needed to maneuver through the environment. 

2. Interfaces 

Training and mission preparation occurs in many locations and in many forms. 
From initial training in classroom type environments to last minute revisions while flying 


to the mission release point, soldiers are continuously planning, revising, and rehearsing 
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the mission at hand. Because of this, the interface for computer training devices must be 
customized to the task to provide the best possible results. However, we don’t know 
what the task to interface relationship is. 

To the user, the interface is the system and therefore the interface must optimize 
system utility while limiting factors that may hinder performance [HIX 93]. Whether it is 
a keyboard, joystick, mouse, data glove, Polhemus device, locomotion device, or some 
other interface, the device must be easy to use and provide the user with the versatility 
needed to maneuver through the environment to provide positive training transfer. 
Anything less will reduce the effectiveness of the training and could foster negative 
training effects. Continued research is needed in this area to determine what type of 
interface is best for simple navigation through a VE. 

3. Fidelity Levels 

As we increase the capabilities of our hardware and software, the definition of 
low, medium, and high fidelity models become more diverse. In the early 1970’s, people 
were impressed with the wonderful new game played on televisions called Pong. Today, 
home gaming systems such as SEGA and Nintendo attempt to dominate the market with 
football, basketball, baseball, and hockey games that have players who look and perform 
like their real world counterparts. What was considered “high” fidelity gaming action in 
the seventies and eighties is now considered low-end computer graphics. 

The closer to reality we approach, the less forgiving the user. The user trusts the 
model to such a degree that any inconsistencies he encounters may cause him to lose 
confidence in the model representation. Most people understand that a map will have 
inconsistencies, especially in the area of vegetation. When these same discrepancies 
appear in a VE, participants seem less willing to accept the shortcomings as minor 
limitations. People forget that a VE is a tool. Instead they often become flustered with 
and untrusting of the model. 

At some point, as a model mimics reality, close is not good enough and the VE 
must be near 100% accurate. However, the point where this occurs is unknown. The 
question is, at what level of fidelity do people stop accepting model shortcomings and 
begin demanding complete assurance? We may even be to the point where people are 


more willing to accept a simple model of elevation data covered with an aerial photo over 
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a model that depicts structures and vegetation which may be realistic but, not completely 
accurate in placement or appearance for the purpose of navigation. 

4. Iterations vs Time Limit 

Often participants and soldiers comment that “if I could only do it again, I know I 
would do it better.” They may be correct. Performance as a function of time may be so 
variable as to be statistically meaningless. Performance may depend more on the number 
of times the individual is able to maneuver through the environment. Participants in this 
experiment who were able to explore the environment looking for the control points and 
then make an additional clean run through the course outperformed those participants 
who were able to go through the course only once (Table 5.1). However, no statistical 


significance could be shown due to the limited number of participants in the group. 


Test I Run VE 2 Run VE Participant 
Group Means | Group Means | Pool Mean 
Score 


Table 5.1. VE Participant Results 











This could be due to the opportunity to correct deficiencies in the mental map 
created as the participant explored the environment on his first trip through the course. 
To determine the validity of this hypothesis, participants could be placed into three 
similar groupings of map, real world & map, and VE & map. Instead of being giving an 
hour, ninety minutes, or two hours to study the environment and plan their route, 
participants would be allowed to go through the environment twice. The first time 
through the environment, the participant would be allowed to explore and plan his route. 
The second time through the environment, the map participant would describe his route 
without the use of a map. A real world participant would walk his planned route using 
his map and a compass. The VE participant would execute his route in the VE using his 
map. Participants would have to navigate or explain their routes with no more than one 


major or two minor errors before being allowed to move onto the evaluation phase. 
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Participants could then be evaluated on the number of repetitions it took for them to pass 


the training phase and the number of errors they committed during the evaluation phase. 


Participants in this experiment were also forced to bundle planning and rehearsal 
into one phase. In order to distinguish between the two, participants should be given ten 
to fifteen minutes to plan a preliminary route through the environment prior to exposure 
to the VE or actual environment. During the training phase participants should be 
allowed to make changes to their initial plan. This will force participants to focus their 
efforts on planning before they conduct training. 

5. Designated Route vs Participant Planned Route 

It has been shown that participants can gain path knowledge of man-made 
environments through the use of VEs [WITM 95]. In this experiment it has been 
demonstrated that less path knowledge (Chapter IV, Section A.3) and survey knowledge 
(Chapter IV, Section A.4) is obtained through exposure to a VE than through the use of 
map study of a high resolution 1,5000 orienteering map. 

What should also be studied is the performance of participants provided with a 
pre-planned route with the performance of participants who have to develop their own 
route. Limiting the task to studying and exploring a predesignated route would reduce 
the workload on the participants. This would provide them with more time to concentrate 
on the task of navigating through the environment instead of planning and navigating. If 
VEs can impart enhanced route knowledge, they could be useful in mission rehearsal of 
specific routes. 

6. Male vs Female 

Past research has suggested that males have better spatial ability and visualization 
than females [ANAS 63] [MACC 74] [LLOY 76] and may be based on genetics 
[TAVR 77]. More recent research indicates that differences in spatial ability based on 
gender are becoming less distinguishable [STUM 89] [WEST 98]. No research has been 
conducted to see if this 1s true for natural environments. 

In this experiment, the VE group had one female participant. On average, her 
scores were better than the mean scores for the VE participants and nearly the same as the 
mean for the entire participant pool (Table 5.2). The female participant had an 


outstanding Normalized Error Score due to her ability to quickly identify her errors and 
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construct viable solutions to resolve her situation. Her Whiteboard results indicate a 
relatively poor exocentric view of the environment. Analysis of her image (Appendix-N, 
Figure N.76) shows a shift of the control points to the west and south of their correct 
position. The remainder of her scores were well within one standard deviation of the 


means for both the VE participants and the entire pool. 


Test Female VE Group | Participant 
Participant | Mean Score | Pool Mean 
Score 


Table 5.2. Female Participant Results 













With only one female in the study, no definitive conclusions can be drawn from her 
performance. However, it brings up the interesting question if gender specific navigation 
_ performance is affected by the use of VEs. 
7. Colorblind vs Non Colorblind 
Past research has neglected the influence of colorblindness on navigation 
performance in man-made or natural environments. Since navigation is usually a very 
visual process, this is a major shortfall. In this experiment, there were two colorblind 


participants, both in the Map Group (M2 and M4). On average their average scores were 


better then the mean scores for the Map participants and the mean for the entire 


participant pool (Table 5.3). 















Participants | Mean Score | Pool Mean 
Score 
|Normalized Map Check Score | S690] 424 | 2.455 


Table 5.3. Colorblind Participant Results 
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With only two colorblind participants in the study, no conclusions can be drawn from 
their performance. However, since most VEs make more demands on the visual senses 
than any of the other senses, it brings up the interesting question of color sensitivity and 
the use of VEs to enhance navigation performance. 

8. Experience Level vs Mental Map Development 

In this and in MAJ Banker’s experiments, much emphasis was placed on the 
experience of the participant. Little research was focused on analytical ability. The 
assumption was made that individuals experienced with orienteering or military land 
navigation would best be suited as participants for this type of land navigation 
experiment. 

One pilot participant for this experiment had less than one week of military 
navigation training and no experience with orienteering. The participant had the lowest 
score on the Map Reading Test (70% correct). He scored the lowest on the Santa Barbara 
Sense-of-Direction Scale (73) and classified himself as a beginner on the Self-Ability 
Evaluation. He did score above the national average on the Guilford-Zimmerman 
Aptitude Survey (26.75) and is a Rhode Sc: ‘ar applicant. On average, his scores were 
better than the mean scores for the VE participants and the entire participant pool for 


landmark and route knowledge scores but were much worse for the survey knowledge 


tasks (Table 5.4). 
Test Novice Pilot VE Group | Participant 
Participant | Mean Score | Pool Mean 
Score 


Landmark Score (Higher score is better) 
Total Errors 
| Normalized Map Check Score 


Table 5.4. Novice Pilot Participant Results 
As stated in Chapter 4.B.2, there is significant correlation between performance 
and the Guilford-Zimmerman Aptitude Survey. Although the inexperienced novice 
participant showed difficulty with the survey knowledge tasks, he performed very well on 


the landmark and route tasks. This indicates there may be a correlation between 
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navigation performance and an individual’s analytical abilities, which is independent of 
experience or training. An individual’s analytical ability may impact on the way they 
organize thoughts and create mental maps. This correlation may lead to the conclusion 
that certain individuals are more prone to being natural Jand navigators. 

9. Medium vs Time of Exposure 

Many military missions and most hostage situations are made more complex by 
time constraints. Often individuals and teams have only days or hours to prepare for 
complex scenarios which require detailed precision. These time lines limit resources 
which can be made available to assist in mission preparations. Research indicates that 
with limited time and exposure, VEs provide very limited performance enhancement and 
may even be counter productive [GOER 98]. 

Due to limited resources and the complexity of the research, many experiments 
involving the usefulness of VEs to assist in the transfer of spatial knowledge to real world 
environments have been limited to exposure durations of less than an hour; often only a 
few minutes. Under these limited exposure times, participants may not be able to resolve 
the differences between the map and the VE or build an uncluttered and continuous 
mental map of the real world. Depending on the size and complexity of the model and 
actual environment, participants may need more exposure time to resolve differences and 
build a valid mental map of the terrain. 

In view of the limited amount of information on a map, the time to resolve 
differences is less than when developing a mental model from a VE. However, a map 
cannot display as much information as a VE nor can it resolve errors that may occur from 
false readings of terrain features, such as mistaking a map depiction of a depression as a 
hill. A map can provide a rough sketch of the environment with simple references that 
can easily be confirmed or refuted by the participant as he moves through the 
environment. If time is limited, a map may give an individual the rough geocentric view 
of the world needed to make simple movements through the terrain. This may be why 
the map only group outperforms the VE and real world groups for limited exposure 
times. The amount of information which can be gleaned from a map is finite and 
influences the limited level of performance one can expect for a first time pass through 


the actual environment. 
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Army doctrine recognizes that terrain is not neutral. The terrain provides a 
distinct advantage to the side which recognizes its limitations and advantages and uses 
this knowledge in the planning and conduct of operations [FM10 93]. In the past, this 
knowledge is gained through constant exposure to the environment that provided the 
owner of the terrain with additional information which could only be learned from on 
sight observations. Depending on the individual, the acquisition of such xnowledge takes 
days, weeks, months, or even years. With limited exposure, individuals may only gain 
route knowledge of their environment and may never gain survey knowledge depending 
on how much of the terrain they are exposed to. 

A virtual environment has many of the same characteristics as the real world with 
the added advantage of being able to look at the environment in many ways that are 
physically impossible in the actual world. Users can view the terrain from any altitude 
and position which allows them to freely move from ego to excocetric views. However, 
a virtual world cannot represent all the actual trees, holes, rocks, and other extraneous 
items in the environment. Geometric models may not be able to keep up with changes in 
- the environment such as new ditches, damaged or modified structures, or fallen trees, due 
to insufficient intelligence reports, last minute corrections, or Haraware and software 
limitations. This prevents the virtual world from being completely accurate in its 
representation of an environment. 

Taking into account the strengths and weaknesses of the media, a chart has been 
developed depicting a possible correlation between land navigation performance and 
exposure to the media (Figure 5.1). The lower left corner represents the expected 
performance of an individual with no prior knowledge of the environment, no 
navigational experience, and no navigational aids who is inserted into the environment 
and told to move from Point A to Point B. The upper right corner represents the optimal 
performance expected of an individual who has been exposed to the environment for an 
extended period of time, possibly years, who is asked to move from Point A to Point B 


with no navigational aids. 
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Figure 5.1. Performance Curves 

The graph indicates an initial superior performance by those who are exposed to a 
map of the environment before insertion. This curve gives way to the high fidelity VE 
with map curve. The low fidelity VE with map curve initially outperforms the high- 
fidelity VE due to the additional time required by the high fidelity VE user to resolve 
differences and differentiate clutter from actual valid terrain features and landmarks. 
However, once these difference are resolved, the high fidelity VE outperforms the low 
fidelity VE curve due to the additional information it can provide. The real world curve 
initially outperforms the VE curves because less time is lost resolving differences in 
mental maps. The VE curves soon pass the real world curves due to the ability to 
compress training times in a VE and to gain both egocentric and excocentric views of the 
environment using the VE. The real world curve plateaus as the user becomes roughly 
familiar with the environment but still has not transitioned into a state of survey 
knowledge of his environment. Once an individual makes the transformation to survey 
knowledge in the real world, the real world curve begins to climb and soon surpasses the 


VE curves in perforrnance. The optimal expected performance of the real world over the 


? 


VE is credited to the VE’s inability to represent all the aspects of the real world. Most 
research in this area has been to the far left side of the time line. Further research is 
needed to validate the graph and curves to determine what the general values would be 
for the time scale and expected general navigational performance levels. 

10. Navigation from Sea to Shore 

Research has been conducted to identify participant abilities to navigate an open 
water virtual environment [DARK 95], a non real-time natural virtual environment 
[BANK 97], an overland helicopter virtual environment [SULL 98], and a real time 
natural virtual environment. However, no research has been conducted to test the validity 
of using a VE to train individuals to conduct from the sea navigation. From the sea 
navigation consists of approaching land from the open ocean or sea and conducting a 
mission on or over the land. This type of navigation requires the individual to transition 
from navigating open waterways to conducting appropriate navigation over land (flying) 
or on land (driving or walking). One of the most difficult tasks in this type of navigation 
is properly identifying terrain features while on the open water to use as guides for 
transition to navigation on land. 

This type of navigation is routinely done by sailors and fishermen coming into 
ports or beach heads, Navy helicopter pilots conducting rescue missions of downed 
pilots, and by Navy Seals and Special Forces Teams who are infiltrating enemy territory 
from the sea. Reducing the chances of misinterpreting a proper land site or transition 
point can reduce parallel navigational errors, reduce mission execution times, and 
increase the chance for « verall mission success. 

11. Mandatory Map Checks at Each Control Point 

One strategy used to help overcome short-term memory issues was to make a map 
check at each Control Point. One participant used this technique, M3. By conducting 
these checks, the participant verified his position within the world and was able to 
quickly memorize the route to the next contro] point. 

Another issue is the reason people make map checks. Map checks can be broken 
down into two basic categories, maintenance checks and recover checks (Appendix Q, 
Figure Q:1). Maintenance checks are conducted to verify or confirm an individual's 


position, orientation, or route. A recover check is performed to determine an individual's 
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position, orientation, and plan a new route to return to the desired location. The 
difference for why an individual makes a map check drastically impacts on the amount 
and usefulness of information they extract during the check. Individuals, who are 
completely bewildered by the environment, can look at a map for minutes without 
resolving any of the issues and can confuse themselves even more. While individuals 
who know exactly where they are in the world and on the map can quickly verify their 
planned route, make modifications, and resolve any discrepancies in their mental maps. 

Future studies should classify the type of checks an individual is performing to 
better understand how much the individual actual understands the terrain. The short term 
memory issue can be resolved by requiring each participant to conduct a fifteen second 
map check at each Control Point. Care must be taken to ensure the mandatory map 
checks do not turn the experiment into a map reading exercise. The question of map 
check classification can be resolved by asking the individual to tell the experiment 
monitor why they are making a map check. If participants indicate that they are verifying 
their location or route, the check is classified as a maintenance check. If participants 
indicate they are lost, trying to fix their position or planning a new route, then they are 
performing a recover check. 

12. Orienteering Map vs 1:50,000 Map 

Previously we discussed how the fidelity of the map may have played a major role 
in the performance of the Map Group Participants (Chapter 4.B.9). It is true that most 
military navigation is performed on a 1:50,000 (Appendix F, Figure F.1) or 1:24,000 
(Appendix F, Figure F.2) map with rough sketches, blue prints, or aerial photos of the 
target areas. Most military personnel would never be afforded such a high resolution 
map depicting most linear features and categorizing the terrain to the degree that the 
experiment’s 1:5,000 map provided. A more realistic study may use the 1:24,000 map in 
conjunction with an aerial photo and sketches of the control point areas instead of the 
high fidelity orienteering map. Participants could then draw their proposed route on an 
8.5x11 inch aerial photo of the terrain. This would aprile research monitors with the 
same resolution map to track and evaluate participant movement without providing a map 
with such a high degree of fidelity. This could also help to reduce problems with 


resolving distances for participants who are accustomed with standard military map 
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distances. Figure 5.2 shows the potential performance levels of individuals using 
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Figure 5.2. Medium Resolution vs Performance Diagram 

differing training media. The performance curves for real time VEs, non real time VEs, 
and 1:5,000 scale map participants at the one-hour exposure mark are based on the results 
of the Banker experiment [BANK 97] and this thesis. The 1:24,000 and 1:50,000 scale 
map curves at the one-hour mark are predicted results. The optimal exposure times in 
each medium before performance levels off have not been verified through research. 
These times will vary based on the complexity of the environment and the abilities of the 
individual navigators. 

13. Run Route Backwards 

To reduce the memorization requirements and the length of the course, 
participants could be asked to plan and study a route that had only five or six controls. 
To increase the difficulty in the execution of the course while reducing the distance to be 
covered, the participants could then be asked to run the course in reverse order. For 
example, participants would be asked to plan a route through the control points in 
alphabetical order (A, B, C, D, E). When the participants are taken to the course, they are 


then informed they will run the course in reverse order (E, D, C, B, A). 
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This will allow participants in the VE and Real World Groups to have plenty of 
time to complete the course one or more times. It will also test to see if participants can 
play their mental movies or traverse their list of mental directions in reverse order. This 
would help to indicate that survey knowledge is being gleaned from the training phase 
and not merely route knowledge. 

14, Introduction of Secondary Task 

For most military missions, navigation is not the primary task. Instead it is a 
secondary task to get an individual or group of individuals to a location so that the 
primary task can be accomplished. Furthermore, during most military navigation 
operations, the task shares focus with the need to provide security and communicate with 
other entities. During this experiment, participants were asked to concentrate on 
navigating through the course as their primary task. At no time was an individual 
required to conduct more than one-task simultaneously during the execution phase of the 
experiment. 

To provide more validity to the experiment, it would be prudent to quantitatively 
show that after training in a VE, individual navigation performance improved in the real 
world environment while secondary tasks are being performed. The secondary task 
would need to be simplistic in design to ensure additional training is not required to 
perform the task. At the same time, it must be complicated enough to require mental 
resources to be concentrated on the task for its successful completion. 

15. Compare Execution Through VE and Real World 

To better determine if a virtual world can substitute for its real world counterpart, 
an experiment testing navigation performance in a virtual world compared to navigation 
performance in the real world may provide some further insight. Both groups would be 
provided with the same training materials to plan a route through a course. Next, the 
participants would execute their planned routes through the actual navigation course or 
through a VE of the course. Measurements could then be taken on errors, distance per 
error, map checks, Wheel Tests, and Whiteboard Tests to determine if the two groups 
perform at relatively similar levels. This would help to ascertain if VEs are a viable 


substitute to a full-scale mock-up of the actual terrain. 
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APPENDIX A. EXPERIMENT OUTLINE 


1) In Brief/Consent Form 


2) 


3) 


4) 


a) Time —5 Min 

b) Location — CS Student Conf Room 

c) OIC—CPT Simon R. Goerger 

d) Materials - Consent Form, Privacy Act Statement, Minimal Risk Consent Form, 
Subject Roster, pencils, Fort Ord Map (confirm the subject has not been on the 
course terrain before), In Briefing Script 


Color Blindness Test/Self Evaluation Questionnaires/Map Reading Test 


a) Time — 15 Min 

b) Location — CS Student Conf Room 

c) OIC—CPT Simon R. Goerger 

d) Materials — Color Charts (1 min), Self Ability Evaluation Sheet (1 min), Santa 
Barbara Sense of Direction Scale Questionnaire (3 min), Map Reading Test (5 
min), pencil 

e) Grading (5 min) 


Spatial Orientation 


a) Time — 15 Min 

b) Location — CS Student Conf Room 

c) OIC —CPT Simon R. Goerger 

d) Materials — Guilford-Zimmerman Aptitude Tests (10 min), pencils, answer sheets, 
e) Grading and Grouping (5 min) 

f) Groups 


i) Group A - Upper 50 percentile 
ii) Group B - Lower 50 percentile 


Interface Familiarization (VE Only) 


a) Time — 15 Min minimum 

b) Location — Graphics Lab 

c) OIC—CPT Simon R. Goerger 

d) Materials ~SGI machine, Performer Town Model, Flybox instructions, Virtual 
Environment Briefing Script, Interface Familiarization Checklist 

e) Movement (15 min minimum) 
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5) Training 
a) Map Group 


(1) Time — 60 Min 

(2) Location — CS Student Conf Room 

(3) OIC — CPT Simon R. Goerger 

(4) Materials — Fort Ord Orienteering Map, Participant Task List, Map 
Marking Instructions, red alcohol marker, alcohol marker eraser, pencil, 
scratch paper, orienteering clue sheet, Map Group Briefing Script, Training 
Evaluation Sheet 


b) Real World Group 


(1) Time — 60 Min 

(2) Location — Fort Ord Orienteering Course 

(3) OIC — CPT Simon R. Goerger 

(4) Materials — Fort Ord Orienteering Course, Fort Ord Orienteering Map, 
Participant Task List, Map Marking Instructions, red alcohol marker, 
alcohol marker eraser, pencil, scratch paper, orienteering clue sheet, 
compass, Real World Group Briefing Script, Training Evaluation Sheet 


c) Virtual Environment Group 


(1) Time — 60 Min 

(2) Location — Graphics Lab CPT Simon R. Goerger 

(3) OIC — CPT Simon R. Goerger 

(4) Materials — Elvis (SGI ) w/ flybox and 21’7/40” screen configuration or 
projector, Fort Ord Model, Fort Ord Orienteering Map, Participant Task 
List, Map Marking Instructions, Flybox instructions, red alcohol] marker, 
alcohol marker eraser, pencil, scratch paper, orienteering clue sheet, 
Virtual Environment Briefing Script, Training Evaluation Sheet 


6) Testing (est Time 120 Minutes — travel to Fort Ord Orienteering Course, run the 
course, and return). 


a) Time — Travel Time 30 Min (total); Run Course 90 Min; Total Time (120 min) 
b) Location — Fort Ord Orienteering Course 
c) OIC—CPT Simon R. Goerger 
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d) Materials — Clipboard with subject’s map & designated route, compass, Think Out 


e) 


Loud Instructions, Data Collection Sheet, red pen to record data, blue alcohol pen, 
stop watch/timer, Color Wheel for Tasks 3.1. & 5.1, White Board with ten 
magnets, rucksack frame w/GPS system, helmet & camera, water, first aid kit 
(cellular phone), Course Briefing Script, blind fold (for movement to course), 
spare clue sheet & color wheel arrows, Tecnu (for poison oak) 

Tasks: 





i) Task 1. (Path Knowledge) Move from the starting point to Checkpoint #1 
along designated route. (measure elapsed time and number of errors; mark 
deviation from route on map) 

li) Task 2. (Path Knowledge) Move from Checkpoint #1 to Checkpoint #2 along 
designated route. (measure elapsed time and number of errors; mark 
deviation from route on map) 

ii) Task 3.1. (Survey Knowledge) Take bearings to SP, CP #5, and CP #9 at the 
south side of CP #4) 

iv) Task 3.2. (Path Knowledge) Move from Checkpoint #2 to Checkpoint #3 

along designated route. (measure elapsed time and number of errors; mark 
deviation from route on map) 

v) Task 4. (Path Knowledge) Move from Checkpoint #3 to Checkpoint #4 along 
designated route. (measure elapsed time and number of errors; mark 
deviation from route on map) 

vi) Task 5.1. (Survey Knowledge) Take bearings to CP #1, CP #6, and CP #8 at 
the south side of CP #4) 

vii) Task 5.2. (Path Knowledge) Move from Checkpoint #4 to Checkpoint #5 
along designated route. (measure elapsed time and number of errors; mark 
deviation from route on map) 

viii) Task6. (Path Knowledge) Move from Checkpoint #5 to Checkpoint #6 
along designated route. (measure elapsed time and number of errors; mark 
deviation from route on map) 

ix) Task 7. (Path Knowledge) Move from Checkpoint #6 to Checkpoint #7 along 
designated route. (measure elapsed time and number of errors; mark 
deviation from route on map) 

x) Task 8. (Path Knowledge) Move from Checkpoint #7 to Checkpoint #8 along 
designated route. (measure elapsed time and number of errors; mark 
deviation from route on map) 

x1) Task 9. (Path Knowledge) Move from Checkpoint #8 to Checkpoint #9 along 
designated route. (measure elapsed time and number of errors; mark 
deviation from route on map) 

xii) Task 10. (Survey Knowledge) ‘Have subject indicate bearing and route he 
must traverse to make it to Checkpoint #4. Have subject return to 
Checkpoint #4. (mark route and any turn which leads the subject away 
from Checkpoint #4. Allow a maximum of ten minutes to return to 
Checkpoint #4) 
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7) 


xiii) Task 11. (Survey Knowledge) Have subject arrange magnets on the white 
board indicating the location of the starting point and nine checkpoints. 
Measure time and note method of magnet placement (i.e. in order of visit, 
outside-in, or inside-out). Take picture of final results (allow 5 minutes 
maximum). 


g) Error (Definition) 


Subject strays from designated route (5 meters from designated route on a 
path/trail/road; 15 meters from cross country designated route). (record one error) 


Debriefing. 


a) Time — 30 Min 

b) Location — Graphics Lab 

c) OIC — CPT Simon R. Goerger 

d) Materials — Clipboard with subject’s map & designated route, Data Collection 
Sheet, red pen to record data, GPS system, Troop (PC) w/ Arcview and Fort Ord 
Maps, digital camera, Participant Questionnaire(s), Researcher’s Script 

e) Administer questionnaire(s); down load GPS datum and ¢ splay on aerial photo 
using Arcview. 

f) Discuss rout:. 


i) Have the subject complete the Debriefing Questionnaire. Read their answers 
and ask for any clarification. 

il) Walk the subject through his route using the subjects planned route and the 
GPS data down loaded from the Message Pad and plotted on the aerial 
photo in Arcview. 


(a) Have the subject to explain why they deviated from their route at those 
locations where the two differ. 

(b) Have the subject explain when & how they determined they were off 
course. 

(c) Have the subject explain how they recovered. 


iii) Ask the subject if he would have done anything different in the training phase 
now that has completed the experiment. 

iv) How much time does the subject spend playing computer games or working 
with computer graphics (more than an hour a day, a couple hours a week, 
once or twice a month, rarely, never)? 
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APPENDIX B. TASK LISTING 


Task 1. (Path Knowledge) Move from starting point to Checkpoint #1 along 
designated route (measure elapsed time and number of errors; mark deviation from route on 
map). 

Task 2. (Path Knowledge) Move from Checkpoint #1 to Checkpoint #2 along 
designated route (measure elapsed time and number of errors; mark deviation from route on 
map). 

Task 3.1. (Survey Knowledge) Take bearings to SP, CP #5, and CP #9 at the 
south side of CP #4). 

Task 3.2. (Path Knowledge) Move from Checkpoint #2 to Checkpoint #3 
along designated route (measure elapsed time and number of errors; mark deviation from 
route on map). 

Task 4. (Path Knowledge) Move from Checkpoint #3 to Checkpoint #4 along 
designated route (measure elapsed time and number of errors; mark deviation from route on 
map). | 

Task 5.1. (Survey Knowledge) Take bearings to CP #1, CP #6, and CP #8 at 
the south side of CP #4). 

Task 5.2. (Path Knowledge) Move from Checkpoint #4 to Checkpoint #5 
along designated route (measure elapsed time and # errors; mark deviation from route on 
map). | 

Task 6. (Path Knowledge) Move from Checkpoint #5 to Checkpoint #6 along 
designated route (measure elapsed time and number of errors; mark deviation from route on 
map). 

Task 7. (Path Knowledge) Move from Checkpoint #6 to Checkpoint #7 along 
designated route (measure elapsed time and number of errors; mark deviation from route on 
map). 

Task 8. (Path Knowledge) Move from Checkpoint #7 to Checkpoint #8 along 
designated route (measure elapsed time and number of errors; mark deviation from route on 


map). 
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Task 9. (Path Knowledge) Move from Checkpoint #8 to Checkpoint #9 along 
designated route (measure elapsed time and number of errors; mark deviation from route on 
map). 

Task 10. (Survey Knowledge) Have subject indicate bearing and route he must 
traverse to make it to Checkpoint #4. Have subject return to Checkpoint #4 (mark route 
and any turn which leads the subject away from Checkpoint #4. Allow a maximum of ten 
minutes to return to Checkpoint #4). 

Task 11. (Survey Knowledge) Have subject arrange magnets on the white 
board indicating the location of the starting point and nine checkpoints. Measure time and 
note method of magnet placement (i.e. in order of visit, outside in, or inside out). Take 


picture of final results (allow 5 minutes maximum). 
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APPENDIX C. BRIEFING SCRIPTS 
1. GENERAL 


The scripts in the appendix appear in the same format utilized for the experiment and do 
not follow the standard thesis format utilized in the chapters of this document. This appendix 
consists of five briefing scripts: In Briefing, Control Group Briefing, Map Group Briefing, 
Virtual Environment Briefing, and the Course Briefing. Each participant receives the In 
Briefing and Course Briefing. The participants are exposed to either the Control Group 
Briefing, Map Group Briefing, or Virtual Environment Briefing depending on which group they 
are assigned. This appendix also contains the Debriefing hand out. 

2. IN BRIEFING 


Welcome to the Naval Postgraduate School’s Computer Science Department. My name 
1S . Thank you for your assistance with today’s experiment. Today’s experiment 
deals with dismounted navigation in natural terrain. 


This experiment is not a test of your intelligence or performance. Rather, it is an 
evaluation of navigational tools. (For Military Personnel) Your performance will not be 
recorded in your personnel records but is intended for research purposes only. All information 
collected is for academic research only. Prior to starting the experiment you will be asked to 
read and sign a series of consent forms. Upon signing the consent forms, you will take self- 
evaluation, map reading, and spatial orientation exams. After the tests, you will under go a 
sixty-minute train-up period prior to moving to the navigation course. Upon completing the 
course, you will be brought back to Spanagel Hall for a short debriefing. 


If there are no questions, please read and sign this consent form. 
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3. CONTROL GROUP BRIEFING 


In front of you is a map of an orienteering course as well as the actual terrain 
depicted on the map. You also have a clue sheet describing the location of the control 
points and photos of the control points. The map, photos, and terrain are for your use to 
study and plan the route you will be using to navigate the course. 


You have sixty minutes to study the map and terrain. Your planned route must 
navigate you through the nine checkpoints in order. (Show the participant the 
checkpoints in order then point out each checkpoint in the photo.) Beginning at the 
designated starting point, you will go to CP1, then to CP2, then to CP3, ... and finally to 
CP9. The checkpoints are described in the clue sheet provided. You may take the clue 
sheet with you when you go on the course. Before the end of the sixty-minute study 
phase, you will mark your planned route on the map using a red alcohol marker. 


After completing the study phase, you will be escorted back to the starting point to 
run the ~oute you designated on your laminated map. While navigating the course, you 
will no: have the map nor will you be allowed to use a compass. During the execution of 
the course, you may request a thirty seconds map or compass check; or a sixty-second 
map and compass check. You can request as many map or compass checks as you wish, 
but each check will be recorded. If you decide to deviate from your previously planned 
route, you may request the map to mark your newly planned route. 


Do you have any questions before we begin? 
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4. MAP GROUP BRIEFING 


In front of you is a map of an orienteering course. You also have a clue sheet 
describing the location of the control points as well as photos of the control points. The 
map and photos are for your use to study and plan the route you will be using to navigate 
the course. 


You have sixty minutes to study the map. Your planned route must navigate you 
through the nine checkpoints in order. (Show the participant the checkpoints in order 
then point out each checkpoint in the photo.) Beginning at the designated starting point, 
you will go to CP1, then to CP2, then to CP3, ... and finally to CP9. The checkpoints are 
described in the clue sheet provided. You may take the clue sheet with you when you go 
on the course. Before the end of the sixty-minute study phase, you will mark your 
planned route on the map using a red alcohol marker. 


After completing the study phase, you will be taken to the navigation course to run 
the route you designated on your laminated map. While navigating the course, you will 
not have the map nor will you be allowed to use a compass. During the execution of the 
course, you may request a thirty seconds map or compass check; or a sixty-second map 
and compass check. You can request as many map or compass checks as you wish, but 
each check will be recorded. If you decide to deviate from your previously planned route, 
you may request the map to mark your newly planned route. 


Do you have any questions before we begin? 
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5. VIRTUAL ENVIRONMENT GROUP BRIEFING 


Prior to beginning the study phase you will under go a fifteen-minute model 
familiarization phase. This is to help you become comfortable with the model controls 
prior to starting the experiment. The model you will be using for this phase bears no 
resemblance to the actual model to be used during the training phase. You will be 
required to show proficiency with the interface prior to moving on to the terrain model. 


In front of you are the 3-screen configuration, a joystick interface, and a list of 
instructions for the use of the interface (demo controls). Please feel free to explore the 
environment and controls for the next few minutes. When you feel confident with the 
controls, I will walk you through a serious of questions to demonstrate your expertise. 


(Conduct Familiarization Phase; after the participant demonstrates proficiency 
with the interface, load up the terrain model and begin the training phase) 


In front of you is a map of an orienteering course as well as a high fidelity 3-D 
model of the terrain depicted on the map. You also have a clue sheet describing the 
location of the control points as well as photos and screen capture images of the control 
points. The map, photos, and VE are for your use to study and plan the route you will be 
using to navigate the course. 


You have sixty minutes to study the map and VE. Your planned route must 
navigate you through the nine checkpoints in order. (Show the participant the 
checkpoints in order then point out each checkpoint in the photo.) Beginning at the 
designated starting point, you will go to CP1, then to CP2, then to CP3, ... and finally to 
CP9. The checkpoints are described in the clue sheet provided. You may take the clue 
sheet with you when you go on the course. Before the end of the sixty-minute study 
phase, you will mark your planned route on the map using a red alcoho! marker. 


After completing the training phase, you will be taken to the navigation course to 
run the route you designated on your laminated map. While navigating the course, you 
will not have the map nor will you be allowed to use a compass. During the execution of 
the course, you may request a thirty seconds map or compass check; or a sixty-second 
map and compass check. You can request as many map or compass checks as you wish, 
but each check will be recorded. If you decide to deviate from your previously planned 
route, you may request the map to mark your newly planned route. 


Do you have any questions before we begin? 


132 


6. COURSE BRIEFING 

Pick-up participant from the Graphics Lab. 

Move participant to the Fort Ord orienteering course. 
Move participant to start point: 

Brief the participant on animals and ammunition 


“You are at the start point of the Navigation Course. During the experiment, I may stop 
you and ask you to answer questions. You must navigate the nine checkpoints in order. 
Each control point will be identified by a control] point marker (show participant a control 
marker) which you must touch prior to moving to the next control point. Once you touch 
a control marker, I will tell you which marker it is. If it is the correct marker, I will give 
you further instructions. If it is the incorrect marker, I will not say anything other then the 
marker’s number. [ will not stop you unless you attempt to cross the course boundaries 
(show participant the boundaries). You may request the compass for a thirty second 
compass check; the map for a thirty second map check; or the map and compass for a 
sixty second compass and map check. These checks will be recorded and timed by me. If 
you determine that you would like to change your route, you may request the map and a 
blue marker to mark changes to your proposed route. You will have sixty seconds to 
mark your new route. You may request an additional sixty seconds if you deem it 
necessary. You have sixty minutes to make it as far as you can along your planned route. 
From now until completion of the navigation course do not interact with anyone. Before 
you begin, do you have any questions?” 


TASK 1: START POINT TO CHECKPOINT ONE. 


Task: “Your first task is to move from the start point to checkpoint one along your 
designated route.” 


Condition: “Without a map or interaction with anyone move from start point to 
checkpoint one along your preplanned route. If you deviate from the designated route 
you will be allowed to continue your movement unless you attempt to go outside the 
course boundaries. You may deviate 5m from your route, if you are on a trail, or 15m, if 
you are conducting cross-country movement before you are assessed an error. You can 
move back and forth along your route without being assessed an error. If you deviate 
from your path for more then 15 continues minutes and are not make progress towards the 
intended control point, I will stop you, show you your location on the map, and give you 
sixty seconds to mark a new route to the appropriate control point.” 


Standard: . “Do the best you can.” 


“Ready,... Begin.” 
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TASK 2: CHECKPOINT ONE TO CHECKPOINT TWO. 


Task: “Checkpoint one. Your next task is to move from the checkpoint one to 
checkpoint two along your planned route. Conditions and standards are unchanged.” 


TASK 3.1.A, B, C: SPATIAL AWARENESS TEST I. 


Stop timer 
Stop participant at spatial awareness test area. 


“Checkpoint two. Stop, I am going to have you identify the direction to three 
checkpoints.” 


Place the color wheel platform in its base on the south side of checkpoint. 

Task: “Identify the direction to the start point, checkpoint five, and checkpoint nine.” 
Show participant arrows as you state their names. 

Condition: “Given a color coded, 360-degree wheel and three arrows, identify the 
direction to the start point, checkpoint five, and checkpoint nine by placing the appropriate 
arrow in the direction of its checkpoint.” 

Standard: “Unchanged.” 

Record the time it takes the participant to perform the Wheel task and the 
orientation of the participant (looking north, south, east, rotates in the direction of the 
arrows, etc). Once done, photo graph the wheel, remove wheel platform from its stand, 
and have participant continue to checkpoint three. 


TASK 3.2: CHECKPOINT TWO TO CHECKPOINT THREE. 


Task: “Your next task is to move from the checkpoint two to checkpoint three along your 
planned route. Conditions and standards are unchanged. Ready,... Begin.” 


Start timer 
TASK 4: CHECKPOINT FOUR TO CHECKPOINT FIVE. 


Task: “Checkpoint three. Your next task is to move from the checkpoint three to 
checkpoint four along your planned route. Conditions and standards are unchanged.” 
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TASK 5.1.A, B, C: SPATIAL AWARENESS TEST I. 


Stop timer 
Stop participant at spatial awareness test area. 


“Checkpoint four. Stop, I am going to have you identify the direction to three 
checkpoints.” 


Place the color wheel platform in its base on the south side of checkpoint. 

Task: “Identify the direction to checkpoint one, checkpoint six, and checkpoint eight.” 
Show participant arrows as you State their names. 

Condition: “Given a color coded, 360-degree wheel and three arrows, identify the 
direction to checkpoints one, six, and eight by placing the appropriate arrow in the 
direction of its checkpoint.” 

Standard: “Unchanged.” 

Record the time it takes the participant to perform the Wheel task and the orientation 
of the participant (looking north, south, east, rotates in the direction of the arrows, 
etc). Once done, photo graph the wheel, remove wheel platform from its stand, and 
have participant continue to checkpoint five. 

TASK 5.2: CHECKPOINT TWO TO CHECKPOINT THREE. 


Task: “Your next task is to move from the checkpoint four to checkpoint five along your 
planned route. Conditions and standards are unchanged. Ready,... Begin.” 


Start timer 


TASK 6: CHECKPOINT FIVE TO CHECKPOINT SIX. 


Task: “Checkpoint five. Your next task is to move from the checkpoint five to 
checkpoint six along your planned route. Conditions and standards are unchanged.” 


TASK 7: CHECKPOINT SIX TO CHECKPOINT SEVEN. 


Task: “Checkpoint six. Your next task is to move from the checkpoint six to checkpoint 
seven along your planned route. Conditions and standards are unchanged.” 
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TASK 8: CHECKPOINT SEVEN TO CHECKPOINT EIGHT. 


Task: “Checkpoint seven. Your next task is to move from the checkpoint seven to 
checkpoint eight along your planned route. Conditions and standards are unchanged.” 


TASK 9: CHECKPOINT EIGHT TO CHECKPOINT NINE. 


Task: “Checkpoint eight. Your next task is to move from the checkpoint eight to 
checkpoint nine along your planned route. Conditions and standards are unchanged.” 


TASK 10.1: CHECKPOINT 4 IDENTIFICATION. 


While standing at checkpoint nine: 


Stop timer 


Task: “Checkpoint nine, finish point. Your next task is to identify the location of 
checkpoint four from where you are.” 


Condition: “Point to checkpoint four and tell me where checkpoint four is from here. 
(i.e., twenty meters and in this direction).” 


Standard: “Unchanged.” 


TASK 10.2; DESCRIBE ROUTE FROM CHECKPOINT NINE TO START 
POINT 


Task: “Your next task is to describe what you consider the easiest route you would take 
to move from here to checkpoint four.” 


Condition: “Without a map, describe the route you would take to move from checkpoint 
nine to checkpoint four.” 


Standard: “Unchanged.” 


TASK 10.3: CHECKPOINT NINE TO START POINT (if described route would 
take them in the general location of the start point) 


Task: “Your next task is to move from checkpoint nine to checkpoint four using the route 
you just described.” 


Condition: “Again, do not interact with anyone to include the researcher. You may not 
request a map or a compass check.” 


Standard: “You have ten minutes, otherwise standards are unchanged.” 
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“Ready,... Begin” 
Start timer 


Reach checkpoint #4 or ten minutes has elapsed. 


FINISH 


Stop timer 


“Stop. Congratulations you have completed the navigation portion of this experiment. 
We will now return to the vehicle for one final test before returning to the laboratory.” 


TASK 11: WHITE BOARD TEST. 


Task: “Your final task is to create a top down representation of the start point and nine 
control points.” : 


Condition: “Without a map or interaction with anyone take the ten magnets labeled with 
the start point and nine checkpoints (show the participant the magnets) and place them 
on a clean white board in proper perspective to each other. You are attempting to create 
. a top down view of the checkpoints, actual distance between points does not matter, 
however, relative locations to each checkpoint does. Until you feel you are finished or 
five minutes has elapsed, you may place and move the magnets as you wish. 

Standard: “Do the best you can.” 


“Any questions,... Ready,... Begin.” 


Start timer 
Stop the timer when the participant indicates he has finished or ten minutes has 
elapsed, which ever occurs first. Observe the participant and note his method for 


placing the magnets (i.e. in order of visit, outside in, or inside out). Take a picture of 
the final results (allow participant 5 minutes maximum to perform the task). 


Stop timer 


“Stop. Congratulations on completing the final task for this experiment. We will now 
return to NPS for a final debriefing session.” 


Move participant back to the Graphics Lab for debriefing. 
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7. DEBRIEFING 


The use of virtual environments in training and education has been an expanding 
field for the last two decades. With recent developments in computer systems, virtual 
reality models are now able to display much higher fidelity. In order to insure we are 
providing a positive training transfer and properly replicating real world environments, 
research is being conducted in the levels of detail required in models. 


The study you have just completed is concerned with gathering information on 
how individuals navigate through complex virtual environments. You spent a session 
planning and studying a route demonstrating route knowledge. Finally, you demonstrated 
spatial knowledge of the terrain through estimating bearings to known points and 
movement to an unplanned location. 


Three separate groups were examined in order to determine performance levels. 
All three groups were given an orienteering map on which they designated their routes 
prior to running the navigation course. The first group was only allowed to study the map 
for 60 minutes. The second group was given the map and allowed to move through the 
terrain for 60 minutes prior to running the course. The third group was give the map and 
was allowed to maneuver through a real time, high fidelity virtual representation of the 
terrain for 60 minutes. 


The research personnel observed and recorded information based on the 
experience and behavior of the participants in order to gather the information equipped for 
the redesign and implementation of a more useful virtual model. The notes and 
observations collected will be used for the purpose of establishing standards for model 
development. 


Your assistance in this project will contribute to the production of more useful 
virtual environments that provide users with spatial knowledge and better navigational 
skills. With the information gathered from your experience and the experience of other 
participants, we are discovering what people generally use as navigational cues in the 
virtual and real world environments. This information will assist in the design of future 
virtual reality models that will be adaptive to a variety of individual needs. 


If you have any questions about the study, please ask your research assistant. 


Until 30 July 1998, please do not discuss this experiment with_anyone except our 
research personnel to prevent influencing any future participants. Thank you for your 


participation in this study. 


The research supervisor, CPT Simon R. Goerger, for this study can be contacted at 
(408) 656 — 4077 or Email: srgoerge@cs.nps.navy.mil. 
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APPENDIX D. CONSENT FORMS 


1. GENERAL 

The forms in the appendix appear in the same format utilized for the experiment and do 
not follow the standard thesis format utilized in the chapters of this document. This appendix 
consists of three documents: Consent Form, Minimal Risk Consent Statement, and the Privacy 
Act Statement. Each participant is required to read and sign these documents before he is 
allowed to participate in the study. A research monitor observes and verifies the signing of 
each document. The format and content of these documents is based on the forms used in 


MAJ William Banker’s land navigation experiment [BANK 97]. 
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. CONSENT FORM 


PARTICIPANT CONSENT FORM 


. Introduction. You are invited to participate in a study of spatial awareness of natural 
and virtual environments. With information gathered from you and other participants, 
we hope to discover insight on navigational aids used to move through virtual 
environments during dismounted navigation of natural terrain. We ask you to read and 
sign this form indicating that you agree to be in the study. Please ask any questions you 
may have before signing. 


. Background Information. The Naval Postgraduate School NPSNET Research Group 
is conducting this study. 


. Procedures. If you agree to participate in this study, the researcher will explain the 
tasks in detail. There will be two sessions: a) 30 pretest phase and 2) training and 
execution phases lasting approximately five hours in duration, during which you will be 
expected to accomplish a number of tasks related to navigating natural terrain. 


. Risks and Benefits. This research involves no risks or discomforts greater then those 


encountered in ordinary hike through rolling, wooded terrain. The benefits to the 
participants are gaining techniques for enhancing spatial knowledge of unfamiliar 
environments and contributing to current research in human-computer interaction. 


. Compensation. No tangible reward will be given. A copy of the results will be 
available to you at the conclusion of the experiment. 


. Confidentiality. The records of this study will be kept confidential. No information 
will be publicly accessible which will possibly identify you as a participant. 


. Voluntary Nature of the Study. If you agree to participate, you are free to withdraw 
from the study at any time without prejudice. You will be provided a copy of this form 
for your records. 


. Points of Contact. If you have any further questions or comments after the completion 
of the study, you may contact the research supervisor, CPT Simon R. Goerger, at 
(408) 656 — 4077 (Email: srgoerge @cs.nps.navy.mil). 


. Statement of Consent. I have read the above information. I have asked all question 
and have had my questions answered. I agree to participate in this study. 


Researcher’s Signature Date 
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3. MINIMAL RISK CONSENT STATEMENT 


NAVAL POSTGRADUATE SCHOOL, MONTEREY, CA 93943 
MINIMAL RISK CONSENT STATEMENT 


Participant: WOLUNTARY CONSENT TO BE A RESEARCH PARTICIPANT IN: Virtual 
Environments and Navigation in Natural Environments 


IE 


I have read, understand and been provided "Information for Participants" that provides the 
details of the below acknowledgments. 


I understand that this project involves research. An explanation of the purposes of the 
research, a description of procedures to be used, identification of experimental procedures, 
and the extended duration of my participation have been provided to me. 


I understand that this project does not involve more than minimal risk. I have been informed 
of any reasonably foreseeable risks or discomforts to me. 


I have been informed of any benefits to me or to others that may reasonably be expected from 
the research. 


I have signed a statement describing the extent to which confidentiality of records identifying 
me will be maintained. 


I have been informed of any compensation and/or medical treatments available if injury 
occurs and is so, what they consist of, or where further information may be obtained. 


I understand that my participation in this project is voluntary, refusal to participate will 
involve no penalty or loss of benefits to which I am otherwise entitled. I also understand that 
I may discontinue participation at any time without penalty or loss of benefits to which I am 
otherwise entitled. 


I understand that the individual to contact should I need answers to pertinent questions about 
the research is Rudy Darken, Ph.D., Principal Investigator, and about my rights as a research 
participant or concerning a research related injury is the Modeling Virtual Environments and 
Simulations Chairman. A full and responsive discussion of the elements of this project and 
my consent has taken place. 


Medical Monitor: Flight Surgeon, Naval Postgraduate School 


Signature of Principal Investigator Date 
Signature of Volunteer Date 
Signature of Witness Date 
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4. PRIVACY ACT STATMENT 


NAVAL POSTGRADUATE SCHOOL, MONTEREY, CA 93943 
PRIVACY ACT STATEMENT 


1. Authority: Naval Instruction 


2. Purpose: Svatial Cognition information will be collected to enhance knowledge, or 
to develop tests, procedures, and equipment to improve the development of Virtual 
Environments. 


3. Use: Spatial Cognition information will be used for statistical analysis by the 
Departments of the Navy and Defense, and other U.S. Government agencies, 
provided this use is compatible with the purpose for which the information was 
collected. Use of the information may be granted to legitimate non-government 
agencies or individuals by the Naval Postgraduate School in accordance with the 
provisions of the Freedom of Information Act. 


4. Dijsclosure/Confidentiality: 


a. I have been assured that my privacy will be safeguarded. I will be assigned a 
control or code number which thereafter will be the only identifying entry on 
any of the research records. The Principal Investigator will maintain the cross- 
reference between name and control number. It will be decoded only when 
beneficial to me or if some circumstances, which is not apparent at this time, 
would make it clear that decoding would enhance the value of the research data. 
In all cases, the provisions of the Privacy Act Statement will be honored. 


b. I understand that a record of the information contained in this Consent Statement 
or derived from the experiment described herein will be retained permanently at 
the Naval Postgraduate School or by higher authority. I voluntarily agree to its 
disclosure to agencies or individuals indicated in paragraph 3 and I have been 
informed that failure to agree to such disclosure may negate the purpose for 
which the experiment was conducted. 


c. I also understand that disclosure of the requested information, including my 
Social Security Number, is voluntary. 


Signature of Volunteer Name, Grade/Rank (if applicable) DOB SSN Date 


Signature of Witness Date 
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APPENDIX E. QUESTIONNAIRES AND TESTS 
1. GENERAL 


The items in the appendix appear in the same format utilized for the experiment 
and thus do not follow the standard thesis format utilized in the chapters of this 
document. This appendix consists of eight documents: Land Navigation Questionnaire, 
Self Ability Evaluation, Santa Barbara Sense-of-Direction Scale, Map Reading Test, 
Guilford-Zimmerman Aptitude Survey, Practice Model Test, and two Debriefing 
Questionnaires. 

The Land Navigation Questionnaire (Appendix E.2) provides a very general 
background of the participant. The participant, prior to arriving to the experiment site, 
completes this questionnaire. 

The Self Ability Evaluation (Appendix E.3) is a qualitative self analysis of an 
individual’s navigational ability. It provides a participant with general limits from which 
to appraise his perceived navigation aptitude. The left end of the scale is valued at 0.00 
and the right end of the bar line is valued at 1.00. Values measured from 0.00 to 0.33 are 
assessed as beginning navigators. From 0.33 to 0.66 is ranked as an intermediate 
navigator. Values of 0.66 to 1.00 are evaluated as experts. 

The Santa Barbara Sense-of-Direction Scale (Appendix E.4) is a quantitative self- 
evaluation of navigational ability. The University of California at Santa Barbara 
developed the scale. An individual’s score is calculated by reversing the values of 
questions 2, 6, 8, 10, 11, 12, 13, and 15. For example, is the participant answered 
question number two as “3”, the question is given a numerical value of “5”. Once the 
values for the above questions are reversed, sum the value of each question and divide the 
total by the number of questions answered. The lower the resulting score the more 
confidant an individual is in their navigational abilities. The University of California at 
Santa Barbara calculates scale’s mean score of 3.54 with a standard deviation of 1.03. 
For this experiment, the mean score was 2.62 with a standard deviation of 0.57. 

The Map Reading Test (Appendix E.5) is comprised of twenty questions dealing 
with terrain feature identification. The test is designed to determine if an individual can 
read the terrain features on a map and associate them to real world terrain features. The 


first fifteen questions relate to properly naming terrain features from 1:50,000 scale 
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military maps and an orienteering map. The last five questions dealt with associating 
images of terrain features to map depictions of terrain features. The answers for the test 
are listed in Table E.1. Each question is worth one point. If a participant misidentifies a 
linear terrain feature they receive 0.5 points for the question. For example if the terrain 
feature is a stream and the participant classifies it as a road, they receive 0.5 points for the 
question. However, if the participant describes a stream as a draw, they receive no credit. 
Participants must score 65% (13 out of 20) or better to be allowed to participate in the 


study. Scores ranged from 13 to 20 with a mean score of 87.5%. 
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The Guilford-Zimmerman Aptitude Survey (Appendix E.6) assesses an 
individual’s spatial orientation ability. The results of this test are compared to a pool of 


national test scores to determine if a participant is above or below the national average 
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for spatial orientation. These results were used to determine which training phase a 
participant would undergo. 

The Practice Model Test (Appendix E.7) is administered to each virtual 
environment participant prior to moving onto the actual course model. It is used to 
ensure that a participant understands and is able to implement the interface functions. 
Each virtual environment participant was required to complete each task of the Practice 
Model Test. After completing the test, a participant is retested on any functions they 
failed to properly employ until he is able to do so. 

The Debriefing Questionnaires (Appendices E.8 and E.9) are administered prior 
to the final review of the participant’s route. Participants in the Real World and Map 
Only Group received the questionnaire in Appendix E.8.  Viurtual Environment 
participants receive the questionnaire in Appendix E.9 that has an additional page 
containing questions related to the virtual environment and its interface. The questions 
are designed to provide a qualitative analysis of the training materials and course. A five 
point scale (1-5) is used for the questionnaire. The final page of-the questionnaire is 
designed to discover those terrain features an individual deems are most needed in a 
virtual environment from which they are obtaining navigational information. One item is 
deliberately left off the list of possible water features to see if participants are paying 
close attention or simply checking items on the list. This feature is streams/rivers. 


The raw scores from these tests and questionnaires are listed in Appendix O. 
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2. LAND NAVIGATION QUESTIONNAIRE 
Name: Age: Sex: 


Branch of Service: Rank: 


1) Where did you first learn to navigate? 


a) Scouting, Boys/Girls Club 
b) Parents 

c) Friend 

d) ROTC/Academy 

e) Basic Training 

f) Officer Candidate School 
g) Officers Basic Course 

h) Other: 


2) How many years have you been Orienteering/avigating? 


a) less then a year 

b) one year or more 
Cc) two years or more 
d) five years or more 
e) ten years or more 


3) At what level would you classify your navigating abilities? 
a) Novice/Beginner 
b) Intermediate/Average 
c) Expert/Advanced 
4) How many Land navigation or Orienteering courses have you done in the last year? 
5) The land navigation course runs through varying degrees of vegetation and over 
rolling terrain. It will require you to negotiate a distance of no more than three miles 


in one hour. Do you have any physical disabilities that would prevent you from 
executing this task? Yes/No 
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3. SELF ABILITY EVALUATION 


Participant ID: 


The following bar line depicts the navigation ability evaluation of an average 
infantry officer with five years experience. The “X”’ indicates his ability level. 


}————_ -—— 


Knows how to Navigates with 
read a map no errors; 
Rarely looks 
at map 


Place an “X”’ on the line below were you feel your navigational abilities are at this 
time. 


ee 


Knows how to Navigates with 
read a map no errors; 
Rarely looks 
at map 
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4. SANTA BARBARA SENSE-OF-DIRECTION SCALE 
Participant ID: Date: SEX: F M AGE: 


This questionnaire consists of several statements about your spatial and 
navigational abilities, preferences, and experience. After each statement, you should 
circle a number to indicate your level of agreement with the statement. Circle “1” if you 
strongly agree that the statement applies to you, “7” if you strongly disagree, or some 
number in between if your agreement is intermediate. Circle “4” if you neither agree nor 
disagree. 

1. Iam very good at directions. 

strongly agree 1 2 3 4 5 6 7 strongly disagree 
2. Ihave a poor memory for were I left things. 

strongly agree 1 2 3 4 5 6 7 strongly disagree 
3. I am very good at judging distances. 

strongly agree 1 2 3 4 5 6 7 strongly disagree 
4. My “sense of direction” is very good. 

strongly agree 1 2 3 4 5 6 7 strongly disagree 
5. Itend to think of my environment in terms of cardinal directions (N, S, E, W) 

strongly agree 1 2 3 4 5 6 7 strongly disagree 
6. I very easily get lost in a new city. 

strongly agree 1 2 3 4 5 6 7 strongly disagree 
7. ITenjoy reading maps. 

strongly agree 1 2 3 4 5 6 7 strongly disagree 
8. I have trouble understanding directions. 


strongly agree 1 2 3 4 5 6 7 strongly disagree 


(turn over and continue) 
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he: 


Il. 


14. 


15. 


I am very good at reading maps. 

strongly agree 1 2 3 4 5 6 7 strongly disagree 

J don’t remember routes very well while riding as a passenger in a car. 
strongly agree 1 2 3 4 5 6 7 strongly disagree 

I don’t enjoy giving directions. 


strongly agree 1 2 3 4 5 6 7 strongly disagree 


. It’s not important to me to know where J am. 


strongly agree 1 2 3 4 5 6 7 strongly disagree 


. usually let someone else do the navigational planning for long trips. 


strongly agree 1 2 3 4 5 6 7 strongly disagree 

I can usually remember a new route after I have traveled it only once. 
strongly agree 1 2 3 4 5 6 7 strongly disagree 

I don’t have a very good “mental map” of my environment. 


strongly agree 1 2 3 4 5 6 7 strongly disagree 
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5. MAP READING TEST 


The following is a list of terrain features commonly found on military and/or 
orienteering maps. Using the list of terrain features, identify the most predominate 
terrain feature within each circle and place your answer in the space provided. Each 
terrain feature from the list may be used more than once or not at all. 


. Depression 

Draw 

Hul Top 

Lake/Pond 

Ridge Line 

Road/Trail 

. Road/Trail Intersection 
Saddle 

© nur/Finger 
Sueam/River 

. Stream/River Intersection 
Valley 


® ciple ole) eallesolie) hy = 
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Depression 

Draw 

Hill Top 

Lake/Pond 

Ridge Line 

Road/Trail 

Road/Trail Intersection 
Saddle 

Spur/Finger 
Stream/River 
Stream/River Intersection 
Valley 
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Spur/Finger 


J. Stream/River 
K. Stream/River Intersection 


G. Road/Trail Intersection 
L. Valley 


E. Ridge Line 
F. Road/Trail 
H. Saddle 


A. Depression 


B. Draw 
D. Lake/Pond 


C. Hill Top 


r. 


sae 
SH: 31 100w 
ay ne 


= e 
a 


+ al 
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Using the following map representations, choose the best representation for each 
picture displayed below. The map representations are a facsimile of the terrain shown in 
the photos. Some map representations may be used more than once or not at all. 
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6. GUILFORD-ZIMMERMAN APTITUDE SURVEY 





= a4 7 : e s ; = , 
“Sic Part 5/Spatial Orientation Mapes aunodeas woe 
| SST wid exis SETI 0 he Ka Suter 





— ae es Dessnduted by Costong Paychotogasts Press, tii. 
“Le” ae 
Name Date...  £Score —.°. Sex: «mM F 
2 
INSTRUCTIONS. 


This is a test of your ability to see changes in direction and position. In each item you are to note how the 
position of the boat has changed in the second picture from the original position in the first picture. 


(If the prow had risen, instead of 
dropped, the correct answer would 
have been C, instead of D.) 


This is the same alming point 
shown above. Note that the prow 
has dropped below it. 


Here Is Sample item 1. These are the five possible answers to the item. 
These bars represent the boat's La This is the prow (front end) ofa 
prow. | motor boat in which you are riding. 
= This fs the aiming point. It is the 

This is the correct answer. {t shows c ehgc! =e you would sie gpo 
that the prow of the boat has drop- le ii. sighted nghtover thaipem 
ped below the aiming point. | OMe pion. 

rT D 

lene 





eel tem 7 





To work each item: First, look at the top picture and see where the motor boat is headed. Second, ok at the 
hottom picture and note the CHANGE in the boat's heading. Third, mark the answer that shows‘the same change on 
the separate answer sheet. 


i 


Try Sample item 2. 






This also shows that the prow of 
the boat fs to the right of the aiming 


point. So, it is the correct answer. This is the aiming point. 


(If the boat had lurned to the left, 
instead of to the night, the correct 


answer would have been A.) This is the same aiming point. 


The motor boat is now headed to 
the nght of it. 


Sample Item 2 


@ Consulting Psychotogists Press, inc., 3803 E. Bayshore Road, Palo Alto, CA 94303 0039 
98 9796 95 94 87654 


Figure E,1 Guilford-Zimmerman Aptitude Survey Cover Page 
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7. PRACTICE MODEL TEST 
a. Turn to a heading of 360 degrees and begin movement. 
b. Switch to a top down view 
c. Switch to a 15-meter view 
d. Change to run mode 
e. Change to walk mode 
f. Move to the road and take a right 


g. While following the road: 


1) Look-up 
ll) Look down 
111) Look left 


iV) Look Right 
h. Head into town 
1. Stop 
j. What is your heading? 
k. Begin movement. 
1. Run 
m. Slow down and stop at the road sign 
n. Lookto your right. What do you see? 
o. Using the quick view keys, see what is at CP6 


p. Using the hot keys, return to the start point 


iS 


8. DEBRIEFING QUESTIONNAIRES 


a. Map and Real World Group Debriefing Questionnaire 











MAP Hard to Read Easy to Read 
Was the map easy to read? 1 2 3 4 5 N/A 
Hard to Understand Easy to Understand 
Was the map easy to understand? 1 2 3 4 5 N/A 
Were the trails & roads adequately Definitely Not Definitely Yes 
shown on the map? ] 2 3 4 5 N/A 
Were the man made structures Definitely Not Definitely Yes 
adequately shown on the map? ] Z 3 4 5 N/A 
Were the obstacles adequately Definitely Not Definitely Yes 
shown on the map? 1 2 2 4 5 N/A 
Was the vegetation adequately Definitely Not Definitely Yes 
shown on the map? 1 Z 3 4 5 N/A 
Using the map, how difficult was it Easy Very Difficult 
to plan your route? 1 2 3 ~ 5 N/A 
Easy Very Challenging 
How difficult was the course? 1 2 3 4 5 N/A 
Definitely Not Definitely Yes 
Were the control points well marked? I 2 g: 4 5 N/A 
Were the control points located Definitely Not Definitely Yes 
where you expected them? 1 2 3 4 5 N/A 
Had routes been trampled down Definitely Not Definitely Yes 
leading to the control points? ] 2 3 4 5 N/A 
Did you have difficulties remembering Definitely Not Definitely Yes 
your planned route? I Z 3 4 5 N/A 
Definitely Not Definitely Yes 
Did you enjoy this experiment? 1 Z 3 4 5 N/A 
Did you feel the training phase Definitely Not Definitely Yes 
was long enough? I 2 3 4 5 N/A 
Did you feel the training phase Definitely Not Definitely Yes 
was too short? 1 2 3 -{ 5 N/A 
Do you feel the training familiarized Definitely Not Definitely Yes 
you learn the environment? 1 2 5 
Did you feel confident in navigating Definitely Not Definitely Yes 
the terrain without a map or compass? Z 3 4 5 N/A 


Comments: 
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1. Place an "X" next to the items you feel must be replicated in a model that prepares you to navigate an 


actual piece of terrain. 


buildings factory roads dirt roads 
buildings houses roads foot paths 
buildings public buildings roads paved roads 
buildings shacks roads trails 
buildings other roads other 
misc compass obstacles electric lines 
misc road signs obstacles pits/fox holes 
misc rock piles obstacles shallow ditches 
misc sand bags obstacles telephone poles 
misc street signs obstacles towers 
misc the sun obstacles trenches 
misc people obstacles other 
misc animals 
misc sound 
misc other vegetation bushes 
misc other vegetation flowers 
vegetation grass/weeds 
terrain clearings vegetation trees 
terrain depressions vegetation other 
terrain hills 
terrain knolls 
terrain ridgelines water lakes 
terrain spurs/fingers water marsh lands 
terrain other water ponds 
water puddles 
water Swamps 
water other 


2. From the list of ites in question # 1, choose and rank the six items you feel are the most important 
for a Computer model which will be used to prepare an individual to navigate an actual piece of terrain. 


NN WN — 
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b. Virtual Environment Group Debriefing Questionnaire 





MAP Hard to Read Easy to Read 
Was the map easy to read? l 2 3 4 5 N/A 
Hard to Understand Easy to Understand 
Was the map easy to understand? ] 2 5 - 5 N/A 
Were the trails & roads adequately Definitely Not Definitely Yes 
shown on the map? ] A 3 4 5 N/A 
Were the man made structures Definitely Not Definitely Yes 
adequately shown on the map? ] 2 3 4 5 N/A 
Were the obstacles adequately Definitely Not Definitely Yes 
shown on the map? 1 2 5 4 5 N/A 
Was the vegetation adequately Definitely Not Definitely Yes 
shown on the map? l 2 =) 4 5 N/A 
Using the map, how difficult was it Easy Very Difficult 
to plan your route? ] Ds 3 + 5 N/A 
Comments: 
Easy Very Challenging 
How difficult was the course? ] Z 3 4 5 N/A 
Definitely Not Definitely Yes 
Were the control points well marked? ] 2 3 4 5 N/A 
Were the control points located Definitely Not Definitely Yes 
where you expected them? ] a 3 4 5 N/A 
Had routes been trampled down Definitely Not Definitely Yes 
leading to the control points? l 2 3 4 5 N/A 
Did you have difficulties remembering 7 a Sie ae a 
your planned route? 
Comments: 









Definitely Not Definitely Yes 










— you enjoy this experiment? ] 2 > 

Did you feel the training phase Definitely Not Definitely Yes 
was long enough? ] 2 3 4 5 N/A 

Did you feel the training phase Definitely Not Definitely Yes 
was too short? ] pe 3 4 5 N/A 

Do you feel the training familiarized Definitely Not Definitely Yes 
you learn the environment? ] Z 3 4 5 N/A 


Did you feel confident in navigating Definitely Not Definitely Yes 





the terrain without a map or compass? ] 2 3) N/A 
Comments: 
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MODEL Definitely Not Definitely Yes 


Was the model clear and viewable? l 2 3 4 5 N/A 
Definitely Not Definitely Yes 

Did the model coincide with the map? l 2 3 4 5 N/A 
Were the trails & roads adequately Definitely Not Definitely Yes 
represented in the model? l 2 3 4 5 N/A 
Were the man made structures Definitely Not Definitely Yes 
adequately represented in the model? ] Z 3 4 5 N/A 
Were the obstacles adequately Definitely Not Definitely Yes 
represented in the model? l 2 3 4 5 N/A 
Was the vegetation adequately ‘Definitely Not Definitely Yes 
represented in the model? l 2 3 4 5 N/A 
Were changes in elevation adequately Definitely Not Definitely Yes 
represented in the model? l 2 3 4 5 N/A 
Did the model help you identify the Definitely Not Definitely Yes 
control points within the last 50m? l 2 5 4 5 N/A 
Did the model help you identify the Definitely Not Definitely Yes 
general area of the control points? l 2 3 4 5 N/A 
Using the model, how difficult was it Easy Very Difficult 
to plan your route? ] 2 3 + 5 N/A 


Do you feel the model gave you an i Definitely Yes 
advantage you normally wouldn't 5 
have had? 

Would you use this tool if it were i Definitely Yes 
available for mission planning? 5 


Would you use this tool if it were —_ Definitely Yes 
available for mission rehearsal? 5 N/A 

Would you use this tool if it were ae he 
available for navigation training? 










IMODELINTERFACE ~——«*Y|Confusing i (<ststi(<‘é;é*~*~*~;~; #Cser Frienddiy:— Friendly 
l 2 3 4 5 N/A 








Were you able to easily move through 
the model? 


Confusing User Friendly 

Was the joystick easy to use? l 2 3 4 5 N/A 
Confusing User Friendly 

Was the acceleration lever easy to use? l 2 3 4 5 N/A 
Confusing User Friendly 

Were the toggle buttons easy to use? l 2 3 4 5 N/A 
Confusing User Friendly 

Your overall felling about the interface? l 2 3 4 5 N/A 
Was the 15-minute train-up on the Definitely Not Definitely Yes 
initial model useful? l 2 3 4 5 N/A 


Was the 15-minute train-up on the Definitely Not. Definitely Yes 
initial model enough time to become l 2 5 N/A 





familiar with the interface? 


Did the use of three screens cause ia _ ae ie 
any confusion when maneuvering? 
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1. Place an "X" next to the items you feel must be replicated in a model that prepares you to navigate an 


actual piece of terrain. 


buildings factory roads dirt roads 
buildings houses roads foot paths 
buildings public buildings roads paved roads 
buildings shacks roads trails 
buildings other roads other 
misc compass obstacles electric lines 
misc road signs obstacles pits/fox holes 
misc rock piles obstacles shallow ditches 
misc sand bags obstacles telephone poles 
misc street signs obstacles towers 
misc the sun obstacles trenches 
misc people obstacles other 
misc animals 
misc sound 
misc other vegetation bushes 
misc other vegetation flowers 
vegetation grass/weeds 
terrain clearings vegetation trees 
terrain depressions vegetation other 
terrain hills 
terrain knolls 
terrain ridgelines water lakes 
terrain spurs/fingers water marsh lands 
terrain other water ponds 
water puddles 
water swamps 
water other 


2. From the list of items in question # 1, choose and rank the six items you feel are the most important 
for a computer model which will be used to prepare an individual to navigate an actual piece of terrain. 


NM & WN — 


160 


APPENDIX F. COURSE 
1. GENERAL 


This appendix consists of six items: 1:50,000 map excerpt of the course area, 
1:24,000 map excerpt of the course area, an aerial photo of the course, an aerial photo 
with an example participant debriefing route, 1:5,000 course orienteering map, and an 
explanation of the map legend [BANK 97]. The 1:50,000 and 1:24,000 maps are the 
standard scales used by most US ground forces for military operations. Comparison with 
the course orienteering map show the magnitude of the additional information which can 
be gleaned from the orienteering map as compared to even the high resolution 1:24,000 
military operations map. The aerial photo is the same one utilized by MAJ Banker to 
produce the original course map and was also used to display the participant’s route 
during the debriefing phase of the experiment. The course map was modified from the 
original one developed by MAJ William Banker after field verification by CPT Simon 
Goerger. The map legend explanation is taken directly from Appendix D of MAJ 
Banker’s 1997 Masters Thesis. 

2. 1:50,000 MAP EXCERPT OF COURSE AREA 

The center of Figure F.1 1s the course area. The boundary roads and two north 
south trails are the only liner features that can be depicted on this map for the area. The 
high ground in the southwest corner of the course and the low ground on the east edge of 
the course are the only discernable terrain features. The entire course is depicted as being 
wooded. A 1:50,000 map of the Fort Ord training area was used to verify that a 
participant had not been in the target area in the past. Its lack of detail and the general 
overview it provided of the training area made it possible to identify locations where 
participants may have explored the old Fort Ord training area without furnishing 


participants additional information about the orienteering course. 
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3. 1:24,000 MAP EXCERPT OF COURSE AREA 

The center of the Figure F.2 is the course area. The boundary roads and two north 
south trails are the only liner features that can be depicted on this map. The high grounds 
in the southwest corner and east of the course are discernable terrain features as well as 
the low ground on the east edge of the course and the northwest corner of the course. 


The entire course is depicted as being wooded. 





Figure F.2 1:24,000 Map Excerpt of Course Area (Actual Size) 
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4. AERIAL PHOTO 
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5. AERIAL PHOTO WITH PARTICIPANT ROUTE 
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Figure F.4. Aerial Photo With Subject Route 
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6. COURSE MAP 
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Figure F.3. Course Map 
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7. COURSE MAP LEGEND EXPLANATION 

All maps are generalizations. They use symbols to portray actual features on the 
earth’s surface. Not all features are represented with the same precision. Discrete non- 
vegetation items are plotted on the map in the exact location they are in the actual 
environment, whereas vegetation boundaries (unless indicated with a distinctive dotted 
line) are not meant to represent a clean break from one type of vegetation to another. 
Rather, this line separating one vegetation area from another is a generalization of where 
one type more or less ends and another more or less begins. The line separating the two 
can best be thought of as a blurry line where the two types of vegetation intermingle. The 
below guide will help to determine the specific limitations of each symbol on the 
orienteering map. 

Building - Buildings in the area are of several types: 

a. Latrines - most common building, tan in color, approx. size 3 x 8 meters 

b. Shelters - second most common building, green wood, roofed, no walls, 

approx. size 3 x 8 meters 
c. Admin. - field office and shack, black with gold trim, 8 x 8 meters and 2 x 2 
meters respectively 

Open Sandy Ground - a significant patch of sand that will slow running 

Open ground - dirt, hard pack, free of grass and other vegetation. 

Undergrowth walk - immature chaparral or oak, dense stands of bushes, 
incomplete overlap of two distinct areas of fight which allow restricted passage along that 


overlap, other plants that prevent running. 
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Fight - mature chaparral or immature oak in such density that passage through is 
very difficult, running impossible 

Forest walk - oak forest with patchy undergrowth, low lying tree limbs or tree 
density that prevents running from being sustained 

Forest slow run - oak forest fairly free of undergrowth, but with low lying limbs 
or tree density that makes sustained running difficult. 

Rough open ground - grass covered ground, possibly with scattered (avoidable) 
undergrowth. Note that there are a few locations that have what appears to be old jeep 
trails but are portrayed as rough open ground. Sometimes the distinction between one or 
the other blurs. If in doubt refer to other more distinctive features (contour lines, etc.) to 
determine your location. 

Shallow depression - most likely an old decaying foxhole position or other man | 
made excavation where the banks have eroded to create a bowl-like depression of 1 to 3 
feet below surrounding ground. 

Misc. object - a manmade feature, rubble, derelict military equipment, or other 
item whose exact description is only provided if it is the location of a control 

Pit - an old foxhole or likely other man made pit that has steep vertical walls and 
may be reinforced with wood, depth from 2 to 5 feet. Note that there will be many pits in 
the area that are not depicted on the map. The pits that are depicted are accurate. 

Telephone poles - wood poles (if bearing wire it will be noted on map) approx. 
25 to 30 feet in height 

Concrete pad - old concrete tent pad extending from 2 to 5 inches above ground 


level 
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Tree - a tree or large bush (could be two or more trees growing close together — 
forming an unbroken single canopy -- if the trees are small) 

Rootstock - a dead or overturned tree 

Troop training device - a bunker or other man made item built for training 
soldiers 

Vegetation boundary - the edge of a vegetation type 

Gully or Ditch - ranging from a shallow 1-foot deep gully to 5-foot deep military 
trench 

Jeep Trail - a road more suitable for 4 x 4 vehicles due to width restriction and/or 
ruts. May be distinctive and worn or in some places overgrown with grass but still 
containing ruts. 

Paved Road - a surfaced all weather road 

Road - a sandy or dirt road wide and level enough for 2 wheel drive vehicles 

Indistinct Path - a path that is in the process of being overgrown with only 
intermittent marks on the ground that indicate that it was once a well traveled path 

Narrow Ride - a linear break in the forest that may have once been a jeep trail 
but now is overgrown with grass and lacks telltale wheel ruts 


Path - a foot or bike path. 
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APPENDIX G. PARTICIPANT TASK LIST’ 


Thank-you for participating in this study. You will do an Orienteering course 
today. However, there are some important differences to note: 


1. You will be wearing a light pack with DGPS and Newton MSG Pad 130. Its purpose 
is to log your route and act as a data capture device for other actions you may perform. 


2. Before you run the course you will carefully plan your route through the entire course 
(see Important Information on Marking Your Map) 


3. Use this training time to commit the route and course to memory. You are expected to 
do the following on the actual course run: 


a. Navigate without aid of map and compass, utilizing only your memory 
b. Attempt to find all the controls utilizing your planned route 


Summary of objectives 
All Objectives are equally important!! 


Choose the most efficient route based on your abilities 

Minimize the number of map checks you request from the administrator 
Minimize the number of compass checks you request from the administrator 
Minimize the number of map with compass checks you request from the 
administrator 

5. Stay on your planned route 

6. Find all the controls in order (you have 60 minutes to conduct this task) 


| 


e Ifyou need to make a map check then say so and the administrator will give you the 
map for 30 seconds. Additional time can be requested in 30 second increments at the 
additional cost of a map check each. 

e If you need to make a compass check then say so and the administrator will give you 
the compass for 30 seconds. Additional time can be requested in 30 second 
increments at the additional cost of a compass check each. 

e If you need both map and compass then say so and the administrator will give you 
both for 60 seconds. Additional time can be requested in increments of 60 seconds. 


e If you want to change your route announce to the administrator that you are changing 
your route plan. At that point the administrator will hand you the map, compass, and 
blue pen. From the time that he gives you the materials you will have 30 seconds to 
plot the new route. If you need more time then tell him you need more time and you 
will get another 30 seconds. Request additional time as needed but remember that 
one of your objectives is to make as few map checks as necessary. Every 30 seconds 
that you are looking at the map beyond the original 30 seconds for the route 
change counts as a map check. 





' This document is adapted and modified from MAJ Banker’s Masters Thesis [BANK 98] 
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APPENDIX H. MAP MARKING INSTRUCTIONS! 


Pay close attention to how you mark your route, be as precise as the map and pen 
allow. Before your actual run you are expected to preview your map within your group’s 
prescribed context. Mark your planned route using the RED pen. You may correct 
any mistakes you make while planning with the white eraser. Once the planning period is 
up or you elect to finish you will not be allowed to erase any of the red route marks you 
have made. SO BE PRECISE in marking your map, detail does matter. Later during 
the actual course run anytime that you are going to deviate from your planned route you 
must stop: 


. Announce to the administrator that you are changing your route plan. At that point the 
administrator will hand you the map. From the time that he gives you the map you will 
have 30 seconds to plot the new route. If you need more time than tell him you need 
more time and you will get another 30 seconds. Request additional time as needed but 
remember that one of your objectives is to make as few map checks as necessary. Every 
30 seconds that you are looking at the map beyond the original 30 seconds for the 
route change counts as a map check. 


. Take the blue pen and draw in your new route with the same attention to detail that you 
applied or the original route planning in red. 


. Leave your original route on the map. The eraser is provided so that you may make 
corrections to a route as you draw it. Once you finish drawing and begin navigating you 


are not allowed to erase routes, or corrections to planned routes (blue penned routes). 


. You may make as many corrections to your route(s) as necessary while navigating the 
course. 


Importance of detail in map marking and navigation 


You are allowed to deviate from your planned route within the following 
tolerances while still being considered on that route: 


Jeep Trails, Paved Roads, Unpaved Roads, Indistinct Paths, Narrow Rides and 
Paths -- If your marked route is on any of these features you are allowed 5 meters either 


side of the feature and you are still considered as being “on your route". 


All other features -- On all other types of non road/trail terrain you may travel 15 
meters to either side of your marked route and you are still considered as being "on your 
route" 


'This document is adapted and modified from MAJ Banker’s Masters Thesis [BANK 98] 
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APPENDIX I. DIGITAL PHOTOS 


1. GENERAL 

Subjects are provided with a series of digital images of the control points. Map 
and Real World subjects receive the photos displayed in Appendix I.2 while Virtual 
Environment subjects receive the Appendix I.3 photo sets. The photos are furnished in 
color. The photos help to outfit the subject with a stronger grasp of the defining 
landmarks they are searching for. Under conditions that would allow the production of 
such a detailed map of the area, it is feasible to expect that reconnaissance photos would 
be available of these locations. 

The VE participants are also presented with screen capture images of the Control 
Points from the same general direction and distance as the actual photos were taken. This 
provides the VE participants with additional information to assist them in resolving the 


differences between the VE and the real world. 
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2. MAP AND REAL WORLD GROUP PHOTOS 
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3. VIRTUAL ENVIRONMENT GROUP PHOTOS 
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APPENDIX J. COURSE EQUIPMENT CHECKLIST 
Binder Containing: 


Subject’s map & designated route 
Think Out Loud Instructions 
Data Collection Sheet 


Researcher’s Script 


Data Recording: 


_____ blue alcohol pen to record route deviations 
red pen to record data 
____ digital camera 

helmet & 8mm camera 


i aanicmemmenaneemmmnamend 


rucksack frame w/GPS system 


stop watch/timer 


Misc: 


_____ extra battery (8mm camera) 
_____extra cassette (8mm camera) 
____ extra Color Wheels for Tasks 3.1. & 5.1 
extra arrows (color wheels) 
______ extra clue sheet (incase subject looses his/hers) 
____ blindfold (for movement to course) 
_____ cellular phone (optional) 
compass 
____ first aid kit 
_______ Tecnu (for poison oak) 


water 


Prepositioned: 


‘Color Wheel Platform for Tasks 3.1. & 5.1 


Control flags 


Nh 





APPENDIX K. THINK OUT LOUD INSTRUCTIONS’ 


Your thoughts are important to this research. As you navigate the course you 
should be “thinking out loud”. 


As you move through the environment and experience it directly express what 
you are thinking. The mental preconception you had of this environment before you 
stepped into it will now be evaluated by you as you experience the course directly. As 
this image is confronted with direct experience your expectations and plan may be 
confirmed, modified, or refuted. Be sure to talk out loud these thoughts. 


The process of talking out loud and paying close attention to your route will slow 
you down. This is expected and why you are given an hour to finish the course. 


PLEASE SPEAK LOUDLY SO THAT YOUR VOICE WILL BE PICKED UP BY 
THE MICROPHONE 


' This document is adapted and modified from MAJ Banker’s Masters Thesis [BANK 98] 
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APPENDIX L. ROUTE CLASSIFICATIONS 
1. GENERAL 

This appendix consists of five items: route analysis, an explanation of route 
classifications for each leg of the course [BANK 97], route classifications based on a 
LISP Program, participant route classifications based on MAJ Banker’s route 
classification and on the LISP generated routes, and optimal route plan for movement 
from Control Point 9 to Control Point 4. The explanation of route classifications for each 
leg of the course is taken directly from Appendix F of MAJ Banker’s 1997 Masters 
Thesis. 

Route classifications were utilized to categorize the difficulty of an individual’s 
planned routes for comparison to their navigational ability. Routes were classified using 
MAJ Banker’s route classification listing (Appendix L.3) and again utilizing the results 
of a LISP route planning program (Appendix L.4). 

2. ROUTE ANALYSIS 

Participant routes were analyzed for difficulty level and performance. 
Participants’ Leg Error Scores were correlated with ther Leg Difficulty Rating and 
ability level. A simple analysis of ability level to planned route difficulty shows that 
participants with higher GZ Scores and a high-perceived level of navigation ability 


planned simpler routes (Figure L.1). 


Cell Line Chart 

Grouping Variable(s): Bar Eval 
Split By: GZ Ability Group 

Error Bars: + 1 Standard Error(s) 
Row exclusion: Data_smsvd 


Intermediate 


Expert 


Beginner 


8 1 12 14 16 18 2 2.2 2.4 
Cell Mean for LISP Average Planned Route Difficulty Level 





Figure L.1. Group vs LISP Planned Route Difficulty by Guilford-Zimmerman Scores 
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This suggests that individuals with higher spatial ability have the ability to 
recognize desirable landmarks on the course and plan more conservative routes to locate 
those landmarks in route to their objective. Participants who ran short of time during the 
study phase due to becoming disoriented in the training environment or failed to 
maximize the tools they were afforded, hastily planned their final legs which usually 
resulting in an azimuth and distance approach to the problem. A straight distance and 
azimuth usually forced participants to negotiate thickly vegetated terrain, in which they 
became entangled and veered off their intended course, resulting in an increased number 
of errors. 

Figure L.1 suggests that routes generated by a LISP program may provide us with 
the ability to predict routes would be best suited for a group based on the team’s spatial 
abilities and navigational experiance. If participants plan routes, which are more difficult 
than their ability level (Appendix L, Section 4), the chances they will fail to successfully 
execute the planned routes increases while intermediate navigators who plan routes closer 
to the beginner level than the intermediate level also find fewer controls (Figure L.2). 
Advanced navigators plan routes just below the intermediate level and perform very well 


as they plan and operate within their abilities. 













Scattergram 
Split By: Bar Eval 
Row exclusion: Data_smsvd 
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Controls Found = 8.043 - .926 * LISP Average Planned Route Difficulty Level; R‘2 = .026 (Intermediate) 


Figure L.2. Performance Based on Ability and LISP Average Route Difficulty 
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The same results are present when comparing performance based on ability levels 
and ISOM Average Route Difficulty Levels (Figure L.3). 
Scattergram 


Split By: Bar Eval 
Row exclusion: Data_smswi 


—" 
= 


@ Beginner 
O Expert 


A Intermediate 


9 
aee 
3 7 
IL 
ieee 
5 5 
oO 
4 
3 
2 


1 12 14 #16 18 %2 22 24 26 28 
ISOM Average Planned Route Difficulty Level 


Controls Found = 8.041 - 953 * ISOM Average Planned Route Difficulty Level; R“2 = .038 (Intermediate) 





Figure L.3. Performance Based on Ability and ISOM Average Route Difficulty 

Conversely, if a program can generate a preplanned route through an environment 
with respect to mission requirements and individual navigational abilities, we can reduce 
the mission planning time by military personnel and concentrate on mission rehearsal. Of 
course, such routes must be reviewed, modified, and verified by the personnel conducting 
the operation to ensure they understand and feel comfortable with the route. 

Further research needs to be conducted to determine if we can accurately predict 
an individual’s navigational performance based on their abilities and the difficulty level 
of planned routes. This research depends on our ability to evaluate an individual’s 
navigational ability and produce a program that can plan viable routes based on mission 
requirements and terrain characteristics. The LISP program in Appendix L.4 is a rough 
draft attempt which takes into account many of the aspects which must be considered if 
such a route planning tool is to be developed. 

3. BANKER’S ROUTE CLASSIFICATIONS 

What follows is MAJ Banker’s classification of some of the most probable routes 
to a given control and is based on the International Specification for Orienteering Maps 
[INTE 90]. They do not represent the only ways of getting to a control but the most 


likely routes chosen by participants based upon MAJ Banker’s orienteering experience 
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and knowledge of the terrain. The classifications are used as a basis for comparison with 
the routes selected by the LISP Route Selection Program. All controls possess at least 
one beginner’s route. The proportion of handrails to catching features delineates 
intermediate and advanced routes. If there are more handrails as compared to catching 
features then the route is intermediate. The opposite is true if there are more catching 
features to handrails. Utilizing MAJ Banker’s method to classify routes taken by 
participants, if an exact match for a participant’s route could not be found from the below 
list, the route was examined within the context of its use of handrails (including what 
type) and catching features and assigned a route designation. This designation correlates 
v' th the same level of difficulty for the utes on that control (beginner, intermediate, 
advanced) [BANK 97]. 


a. Control 1. 


L. Beginner 


a) Gigling Road west to jeep trail 
b) Jeep Trail south by east by south to building 
C) Control on NW corner of building 


Z Beginner 


a) Watkin's Gate Cutoff to indistinct path. 

b) Indistinct path southwest up hill to jeep trail 
Cc) Jeep Trail west to building 

d) Control on NW corner of building 


3p Intermediate 


a) West through plotted individual trees (catching features) 

b) Handrail rough open ground south to junction indistinct 
path and jeep trail 

c} Jeep Trail west to building (catching feature) 

d) Control on NW corner of building 


4. Advanced 
a) West through plotted individual trees 
b) Follow runnable forest southwest 


C) Try to hit small rough open gap by keeping walkable forest 
to left shoulder 
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d) Use forest fight to west as catching feature if needed 
e) Control on NW corner of building 
f) Use jeep trail for catching feature if control is missed 
a): Advanced 
a) Go straight at control from start 
b. Control 2. 
le Beginner 
a) Jeep trail northwest to building 
b) Follow open ground to west and look for rough open 
clearing going northwest (handrail) 
Cc) Follow rough open clearing northwest looking for pit 
d) Control in pit 
= Intermediate 
a) Jeep trail northwest to building 
b) Go straight at control (WSW) from building 
SP Advanced 
a) Set out on straight line directly for control 
b) Hit open ground and look for building on the right and 
rough open break on the left. (Catching feature) 
Cc) Follow rough open clearing northwest looking for pit 
d) Control in pit 
eC: Control 3. 
i Beginner 
a) Head northwest and get out onto Gigling Road 
b) Take Gigling Road west to jeep trail junction with 
telephone pole 
C) Take jeep trail southeast to convergence of two jeep trails 
d) Head southwest into tree grove looking for control 
(1) Use building as catching feature 
(2) Use open ground to west as backup catching feature 
e) Control hanging from tree limb 
2. Advanced 


187 


d. Control 4. 


I 


a) 


b) 


Cc) 


Head straight at control; use jeep trail prior to control as 
catching feature 
Head southwest into tree grove looking for control 


(1) Use building as catching feature 
(2) Use open ground to west as backup catching feature 


Control hanging from tree limb 


Beginner 


a) 


Head southwesterly and try to get on jeep trail headed in 
same direction 


b) Take jeep trail to junction 

c) Take jeep trail southeast to junction 

d) Take southerly fork to next junction 

e) Take fork to northwest 

f) Once beyond patches of fight leave trail and start looking 
for control 

g) Control ts in pit 

Beginner 

a) Turn around and go back to jeep trail to the east 

b) Take jeep trail southwest to junction 

Cc) Take fork to the south to another junction 

d) Take fork to the west to next junction 

e) Take southerly fork to next junction 

f) Take fork to northwest 

g) Once beyond patches of fight leave trail and start looking 
for control 

h) Control is in pit 

Intermediate 

a) Go south towards road junction 

b) Get on road and take to junction 

c) Take road west to other road junction 

d) Handrail around fight to west coming down through small 
patch of fight into control 

Advanced 
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a) Head straight at control expect to hit jeep trail that runs 
NW to SE (catching feature) 

b) Hit trail and then thread way through scattered fight 

c) Emerge into center of depression and rough open ground, 
(catching feature) look for pit 

d) Control is in pit 


Control 5. 
l. Beginner 
a) Move back out onto jeep trail 


b) Take trail west to trail junction 
C) Take trail WNW up to misc object 


d) From misc. object go straight at control 
Je Intermediate 
a) Move directly at control 
b) Use Gigling Road as catching feature if miss on control 
Cc) Control is in center of clearing 
3 Advanced 
a) Move directly at control 
b) Use southwesterly linear clearing as catching feature 
Cc) Follow clearing NW right into control 
d) Use Runnable forest along Gigling as catching feature in 
case of miss 
Control 6. 
ie Beginner 
a) Move out onto Gigling Road and take it westerly to 
junction with dirt road 
b) Move down dirt road (south) to junction with jeep trail 
c) Take jeep trail to east look for concrete rubble 
d) Move southeast through runnable forest 
e) Look for control on concrete pad 
pap Beginner 
a) Move straight at control and hit jeep trail 
b) Go southwest on Jeep trail to junction with another jeep 
trail 
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c) Take jeep trail westerly and look for concrete rubble 
d) Move southeast through runnable forest 
e) Look for control on concrete pad 
sy Intermediate 
a) Move south to junction of two jeep trails (catching feature) 
b) Handrail jeep trail southeasterly to clearing (catching 
feature) 
C) Handrail clearing to the west 
d) Hit fight going west (catching feature) and move south 
e) Handrail fight (keeping it on right shoulder) into control 
f) Look for control on concrete pad 
4. Advanced 
a) Move straight at concrete rubble (aiming off technique) use 
jeep trail as catching feature and handrail 
b) Move southeast through runnable forest 
C) Look for control on concrete pad 
Control 7. 
ik. Beginner 
a) Move back out onto east west jeep trail 
b) Go west to junction of jeep trail and dirt road 
C) Take dirt road south to junction with four jeep trails 
d) Take jeep trail east by northeast 
e) Look for second linear break in vegetation (indistinct path) 
f) Take indistinct path (handrail) to ditch 
g) Follow ditch to its end 
h) Control at east end of ditch 
2s Intermediate 
a) Move through rough open ground easterly to jeep trail 
(catching feature) 
b) Follow jeep trail (handrail) to junction with other jeep trail 
by building 
Cc) Locate telephone poles and follow wire (handrail) south 
easterly 
d) Hit fight and turn west and follow fight boundary into ditch 
(handrail) 
e) Control at east end of ditch 
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Advanced 


a) 


Move through rough open ground easterly to jeep trail 
(catching feature) 


b) Take jeep trail to curve where it turns east (hand rail) 

Cc) Leave jeep trail and head straight for control use east west 
jeep trail as checkpoint (catching feature) 

d) Aim off to east side of ditch and go southeast (telephone 
wires to east as catching feature to prevent drifting too far 
east) 

e) Use fight as catching feature 

f) Hit fight and turn west and follow fight boundary into ditch 

g) Control at east end of ditch 

4. Advanced 

a) Move straight at control 

b) Use jeep trail junction as attack point 

c) From attack point take offset route to west part of ditch 

d) Follow ditch to east and find control at end of ditch 

Control 8. 
le Beginner 

a) Handrail fight to the east till hitting the jeep trail 

b) Follow jeep trail northerly through intersection to sharp 
curve to the east 

Cc) Once at sharp curve to east turn off trail to west and look 

| for control in clearing 

d) Control located 1n clearing 

De Intermediate 

a) Handrail fight to telephone poles 

b) Take telephone poles NW back to jeep trail junction 

Cc) Follow jeep trails east to next junction 

d) Take jeep trail north 

e) Leave jeep trail and move directly at control 

Sy Advanced 
a) Move directly at control (avoiding forest walk) use jeep 


trail junction as catching feature 
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b) From jeep trail junction aim off to east of control at sharp 
curve to east of jeep trail keeping eyes open for control in 
clearings 

Cc) Use same trail as Beginner route as catching feature (for 
drift) 

1. Control 9 (Finish) 
Ihe Beginner 

a) Move back out to jeep trail just to east of control 8 

b) Take trail south to four way junction with other trails 
(handrail) 

Cc) Take southeasterly running trail to trail fork 

d) Take northeasterly running fork to five way junction 
(handrail) 

e) Take northwesterly running trail keeping eyes open for 
small break in fight to the east (catching feature) 

f) Take indistinct path into clearing and hook to north 

g) Control on east edge of clearing 

ps. Intermediate 

a) Move back out to jeep trail just to east of control 8 

b) Move off trail using rough open to move closer to control 

Cc) Take rough open out onto jeep trail which runs NE to SW 

d) Take trail to junction with North South jeep trail 

e) follow jeep trail looking for indistinct path 

f) Take indistinct path into clearing and hook to north 

g) Control on east edge of clearing 


3. Advanced 


a) 
b) 
C) 
d) 
€) 
f) 


Move straight at contro] on east by northeast azimuth 

Use trail as catching feature 

Fight to north and south of route used as catching features 
Locate opening in fight 

Take indistinct path into clearing and hook to north 
Control on east edge of clearing 


4. LISP PROGRAM ROUTE CLASSIFICATION 


This program plans a route through a specified piece of terrain based on 


identifiable decision points and terrain characteristics. The information is manipulated by 


a branch and bound search, pruning heuristics, and terrain classification. The algorithms 


are coded in ANSI LISP. Because of the memory requirements of the search’s stacks and 
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the speed of the processors running the program, each leg was limited to passing through 
a maximum of eight decision points. This limitation forced the rejection of possible 
routes. 

Decision points are identified as any piece of terrain that would logically require 
an individual to make a decision on which direction to move. Although in a natural 
environment, a person on foot can move in almost any direction at any time, it was 
assumed that individuals would not intentionally change direction of movement unless 
they knew where they were and where they wanted to go. For this course, 99 decision 
points were identified. Decision points were associated with neighboring decision points 
based on proximity (Figure L.4). This meant that to traverse the course, the program had 
to link together neighboring decision points into a chain of successive segments to 


complete each leg of the course. 





Figure L.4. Decision Points and Neighbors 


The terrain between neighboring decision points is known as a segment. Each 
segment has a different point value based on a list of characteristics. This program 
utilized four factors (distance, mobility, observation, and difficulty of locating the next 
decision point) to determine segment values. Two additional factors which were not 
incorporated but which would have made the program more accurate are change in 


elevation and terrain revisited. 
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Distance between points is not assigned any additional weight in the program’s 
algorithm. Mobility is based on the difficulty of traversing the terrain. Mobility factors 
came directly from the terrain classification used on the orienteering map. Observation is 
also evaluated based of the orienteering map’s terrain classifications. Observation deals 
with the ability to see through or over the terrain’s vegetation. The final factor addressed 
is the issue of identifying when an individual has reached the decision point. Some 
decision points are easier to locate than others are. For example, it is more difficult to 
locate a control point placed in a pit as compared to a black shed in the middle of a 
clearing. The weights assigned for each of these factors was dependent on the ability 


level of the navigator (Tables L.1, L.2, and L.3). 
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Table L.1. LISP Program Mobility Weights 
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Table L.2. LISP Program Observation Weights 
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Table L.3. LISP Program Identification Weights 




























Weights were based on the impact of each element to the successful completion 
of a segment for each type of individual. The most difficult condition receives the 
highest weight. The lowest weight, easiest aspect, which could be assigned for any 
element is 1.0. For all individuals traversing the terrain on a road with open visibility to 


an easily identifiable decision point was weighted the same, 1.0. Conditions are rank 
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ordered from hardest to easiest and then assigned weights based on their position in the 
table. 

Mobility through the terrain was closely coupled for each group. This is based on 
observations that indicated little difference between ability groups in the level of 
apprehensiveness of individuals who were faced with conducting cross-country 
movement. The most difficult terrain to cross, Fight, was weighted as a 4.0 for a 
beginner, 2.0 for an intermediate and 1.0 for and advanced navigator. These weights 
were based on the difficulty for beginners to maintain their course while traveling 
through fight, since they need constant verification that they are going in the correct 
direction. Intermediate navigators have fewer problems maintaining their course through 
difficult terrain but still require some assurance they are on the right course. Advanced 
navigators are more confident in their abilities, need less reassurance they are on the 
correct course and often plan their route to use catching features to confine their 
movement and halt their forward progress in the proximity of the next identifiable 
decision point. 

Visibility plays much less of a factor for advanced navigators than for 
intermediates or beginners. Intermediate and beginning navigators needs reassurance that 
they are on the right course. This is gained through many cues in the environment most 
of which are visual. Advanced navigators can gain reassurance through many senses 
such as the sound of a creek to the north or the warmth of the sun on the left side of their 
face. Because of this, navigators pay less attention to visual cues enroute to their 
objective as they confirm their position through the use of many input factors. This 
results in reducing the weight of the most cluttered environments to 1.2 for advanced 
navigators while intermediate navigators remain at 3.0 and beginners remain at 4.0. 

Identification of the decision point plays the most crucial part of the segment’s 
value for beginners. If beginning navigators cannot identify when they have reached the 
correct decision point, they often become confused or disoriented. This results in their 
becoming lost and losing confidence in their abilities to determine their location and 
continue their movement in a positive direction. Beginners also have greater difficulty 
choosing and identifying appropriate decision points since they continuously question 


their ability. This results in a greater chance of them misidentifying the correct decision 
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point if it is not an obvious one. For these reasons, the weight for hard to identify 
decision points for beginners is set at 5.0. Intermediate navigators have fewer problems 
choosing and identifying appropriate decision points. Since they do not question 
themselves as often as beginning navigators, they are less likely to incorrectly identify a 
decision point. The weight of hard to identify decision points for intermediate navigators 
is set at 3.0. Advanced navigators filter out much of the “noise” of an environment and 
often choose decision points they can readily identify, ignoring intermediate decision 
point enroute. For this reason they have less difficulty identifying the correct decision 
point. The weight for hard to identify decision points for advanced navigators is set at 
1 

The weights for mobility, visibility, and identification are multiplied with the 
segment’s length. Each segment’s value is based on the resulting product. The route 
with the lowest value for its summed segments is chosen as the best route for that ability 
group. 

The program is designed to locate three optimal paths through the course. One 
Beginner (Figure L.5), one Intermediate (Figure L.6), and one Advanced Course (Figure 
L.7) are calculated and displayed on maps for comparison with participant maps. The 
program also produces a sequential list of decision points or waypoints to traverse in 
order to complete the course. Each leg of a participant’s route is compared to the LISP 
program route legs. If two LISP routes have legs that are the same, the leg is classified as 
the easier of the two routes. If a participant’s planned route between control points is not 
the same as any of the computer program’s planned routes, the participant’s route is 
assigned a classification which is most closely associated with the participant’s route with 


respect to the program algorithm’s defining characteristics. 
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Figure L.5. LISP Beginner Route 
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Figure L.6. LISP Intermediate Route 
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= Fort Ord Land N avigation Map. 





Figure L.7. LISP Advanced Route 

5. PARTICIPANT ROUTE CLASSIFICATION 

The following are the results of the route classifications (Tables L.4, L.5, and L.6) 
of each leg of each participant’s planned route and the overall route rating for each 
participant is based on MAJ Banker’s route classifications (Appendix L.2) and the LISP 
Program’s route classifications (Appendix L.3). Each leg was evaluated as Beginner (B), 
Intestnediate (I) or Advanced (A). The summation of the routes were assessed by 
equating each leg classification with a numerical value (Beginner = 1, Intermediate = 2, 
and Advanced = 3) and summing the value of each leg. The number of legs on the course 
then divided this value. The resulting aggregate was then used to determine the difficulty 
level of the entire route. An “X” in the position of errors committed indicates that a 


participant did not attempt this leg of the route. 


| SP, Se | 3s 
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Table L.6. Participant Route Classifications 
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6. ROUTE FROM CONTROL POINT 9 TO CONTROL POINT 4 

During the execution of the course, participants who made it to the end of the 
course, Control Point 9, were required to describe and execute a route from control Point 
9 to Control Point 4. Figure L.8 shows an example of the most efficient route from 
Control Point 9 to Control Point 4 for a beginning navigator. The program limitations 


allowed the route to run through a maximum of 10 decision points. 


2 Fort Ord Land Navigation Map 
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Figure L.8. LISP Beginner Route from CP 9 to CP 4 
Because of the limitations of the program, the route depicted is more difficult than 


the route chosen by any of the participants who performed this task. The LISP route does 
display characteristics of the routes chosen by the participants. The route travels major 
trails which have been traversed by the participant in the past. For example the route 
departs from Control Point Number 9 and heads south down the trail towards the five star 
intersection. The route then turns to the west and heads down the ridge towards the 
intersection south of Control Point 9. Compared to the route from Control Point 8 to 
Control Point 9, this is the same terrain covered by the Beginning and Intermediate 


navigators. 
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APPENDIX M. DATA COLLECTION WORKSHEETS 


1. TRAINING PHASE DATA COLLECTION SHEET 
























PARTICIPANT ID: eae 
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- Oe Se ee ~~ Session Date: 
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Session Start Time: 
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Number Compass Checks: 


Number Map Checks: 
Number of times subject became "lost" 
Number of times subject went out of bounds or fell off the edge of the model 
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2 EVALUATION ERASE. DATA COLLECTION SHEET 


{Session Date: 
Session Start Time: 













ae to 0 CP 3 — 
Bai Move to CP #4 oS 





Move to CP . 
Move to CP #6 


Move to CP #7 
Move to CP #8 


Move to CP #9 


ar pec LO 


Remarks: 
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APPENDIX N. PARTICIPANT DATA 
1. GENERAL 

Subject data consists of five items: Map with planned route, map with executed 
route, of Wheel Test at Control Point #2, of Wheel Test at Control Point #4, and of White 
Board Test. The errors for deviation from the planned route are located in Appendix O. 
The angle and distance measurements for the Wheel and White Board Tests can be found 
in Appendix O.2 and O.3 respectively. 

The correct representations for the Wheel and White Board Tests are shown in 
Figures N.1, N.2, and N.5 respectively. Examples of digital photos of actual subject 
results for the three tests can be seen in Figures N.3, N.4, and N.6. Subject results for 
these tests will be displayed with the subject’s answers in solid lines or numbers 


superimposed over the actual answers which are displayed in dashed lines or shaded 


numbers. 


Side Towards Checkpoint 


Side Towards Checkpoint 





Figure N.1. Correct Wheel Test CP # 2 Figure N.2. Correct Wheel Test CP # 4 
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Figure N.4. Example Subject Wheel Test CP #4 
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Figure N.6. Example Subject White Board Test 
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2. MAP PARTICIPANT NUMBER 1 
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Figure N.7. M1 Planned Route 


206 





8661 AeW 

WSN 439614809 ‘y UdWIS UeNdeD Ag paydayd pjal4 
VSN JaxUeg qf WEIIIINA Jole W Aq dew 

BILUOPIED ‘PIO 34 





peoy bulb 





paiidxy aul) © 
BINOY MAN $yIaIeSg yOelqns »@ 
a | me \t. a oy ro KE aynoy paynsaxy $,9a/qns —~«_- 
1 atieantattreettn tierra eacllas So RS ME REET ET TN eal ' s-OUOWY Aq UoNeUaO yOa/qns 


Figure N.8. M1 Executed Route 
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Figure N.9. M1 Wheel Test CP # 2 Figure N.10. M1 Wheel Test CP # 4 





Figure N.11. M1 White Board Test 
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3. MAP PARTICIPANT NUMBER 2 
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Figure N.12. M2 Planned Route 
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Figure N.13. M2 Executed Route 


210 





Figure N.14. M2 Wheel Test CP # 2 Figure N.15. M2 Wheel Test CP #4 





Figure N.16. M2 White Board Test 
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4. MAP PARTICIPANT NUMBER 3 








51839 O00} 096 


Wed 
.4S.910) Ul IND) OP] YmouEN 


YR dg Pugsipy| 
(Peagdun) peoy 


Pooy POA, 
#94) door 


¢ 


B66! ACW 


YSN JaBsa0g Y UOWIS UeNded Ag pay2ayYD plaid 


YSN Jayueg ‘ WelljijA sole Aq dew 


AN 


Soon 4 ae 
PVE TEPER SASS THRE cays fg 


23 Ob IMAIO2U, NORMED 
@00'S:) eos dow 


as 
™” 
a 


eS 


wae 


” 


od 
“ 


@ 
ot 
a 


ee i 


ZO 


” 


™~N 


“ 


eloped "PO V4 





006 ova 008 O92 002 09 009 OSS 


UDI JO AUINO 


Apunog uope}Hoa -* 


oui ouotrtars | ANU 


(anoqu0o yg) op ecg, ~ _ % 


iNOwOd ¥ OL“ \_ 
INowod ¥ OS.“ \_” 


ujny 
OHASP Buyuj9s} doouy 
HO JOON 

o9iL 

pod geiu09 

Sojod guoyde|a) 

Ud 

Pogo ISiW 
uojsseidop MmolpUS 








OOS = Oe—CEi“‘tCSE 


puno6 uedo yBnoy 
ur - sQ104 


UNAMOIS - O04 HHH 


nom - 30105 POW 

W613 Sl 

Ie - yprosBiopun ‘|I||I| 
punoi6 ugdo |: 
punos6 Apues uedo fg 
Sulping amg 





spunog go no |||II| 


josjuo7 QC) 
usiuls (0) 
vais VY 
pusia 





TI) peoy Guij6i9 


UBid anoy paysnipy s,joefqng=—=— 
9yNOY pauuUejd s,joefqng—~<# 


Osh oo} og Le) 09 








OSOF YsiUld” O60 HeIS 
puna GUT T= 
‘duaj Bapgg _ 

AMS Kpnojy ESM 
~ eW Jaquinn juedioiueg 











Figure N.17. M3 Planned Route 
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Figure N.18. M3 Executed Route 
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Figure N.19. M3 Wheel Test CP # 2 Figure N.20. M3 Wheel Test CP # 4 





Figure N.21. M3 White Board Test 
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5. MAP PARTICIPANT NUMBER 4 
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Figure N.22. M4 Planned Route 
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Figure N.23. M4 Executed Route 
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Figure N.24. M4 Wheel Test CP # 2 Figure N.25. M4 Wheel Test CP # 4 





Figure N.26. M4 White Board Test 
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Figure N.27. M5 Planned Route 
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Figure N.28. M5 Executed Route 
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SUBJECT DID NOT 
REACH CP #4; 


NO DATA 
COLLECTED 





Figure N.29. M5 Wheel Test CP #2 Figure N.30. M5 Wheel Test CP # 4 





Figure N.31. M5 White Board Test 
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Figure N.32. RWI Planned Route 
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Figure N.33. RW1 Executed Route 
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Figure N.34. RW1 Wheel Test CP # 2 Figure N.35. RW1 Wheel Test CP #4 





Figure N.36. RW1 White Board Test 
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Figure N.37. RW2 Planned Route 


224 





8661 Aew 

VSN 1a8sa09 "Yy UOWIS UeNdeD Aq paxdayD Pjal4 
VSN JaxUeg ‘ef WENA sole W Aq dew 

EIWIOPIED "PIO 34 





paidxg owl, © 
Var SE Be : | F ARCO CE aNoY MEN sees Pelqns @ 

uN <.. OSw eb) NS ea “iu ajnNoYy peynsexy s,yalqns —«_¢- 
a ok BS a ) SEMPRA AF aS va. FAS we ' yOWUOWY Kq udNeJUaHO yafans = 
spunog jo no 

A yooayuD ssedwog 9 de 
yooyo dey 

yooug ssedwog 


PUENTE 





33 Ob (VAIO WU] 1B0QNOD 
000s: HqeBoS dew 





@226°e 


21019 000), 096 006 0os8 608 OG2 002 09 009 Os¢ 00% 09> 00v OSE oe O92 00Z OS+ OoL 09 LY) 09 















nue Min aay ~ beg) Usiuls Sysy Hels 
Aop Buje. dood spunog so 0 [III 
OHAOP Buujosj doo, ¥ UnImols - 8010 SE UW —aure-9 
~ A A ia = AIO ICON * josjuoy ©) p IM 
Weg Pd ~ “ yo¥d 40 JNO ” 2s <oaee A aol a OA e sou04 REY : 
(189405 Ul 1nd) OpjYMouON , ~~ —~ = Aspunog usaeibeA oe Redienss. o ‘ug Meee dwa] Bap 99 

MP d PURS PHI nec ONT SES aelel 7 Sojod ououdejoy © IGA - YrosSopun|! — 
(poacdun) pooy = (anowuod uy $) oul PER ud A punoi6 uedo ves V MS 1e3a|9 ee 

pean POA d nower mes Yolgo 2s1N ~ punoi6 Apues vedo BRS SS 
sory doop ~~ NOW0D 0S,“ \./~  uoissoidapmolwUS Sulping gay pusbhet CMY JOQuUnN juedioiped 


Figure N.38. RW2 Executed Route 
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Figure N.39. RW2 Wheel Test CP # 2 Figure N.40. RW2 Wheel Test CP #4 





Figure N.41. RW2 White Board Test 
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Figure N.42. RW3 Planned Route 
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Figure N.43. RW3 Executed Route 
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Figure N.44. RW3 Wheel Test CP # 2 Figure N.45. RW3 Wheel Test CP # 4 





Figure N.46. RW3 White Board Test 
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8661 ACW 

YSN Ja61809 "y UOWIS UeNdeD Aq paxsayD pray 
YSN 4aqueg gq WEIN 10fe W Aq dew 

PILIOPED PIO V4 








a3 t 
at: ars 


ets ‘ 
east 


Lie y ; aids 
Se MIPEATTD peoy Buy619 





33 Ob (BAIOQUl INOJBOD 
eo0e's:s He5S dew 





URld a1InNoYy paisnipy s,joelqng=—= 
a]noy peuuejd s,joelqns—<— 








$12}8 000! 096 006 «4=— sé 008 002 00z 090 009 ~=—s«os9 009 Oo> 0b ov o0€ sO 00Z sé 00} 09 ry 00 





Ovol YSI4lS §=6ce6O HEIS 





uIny 3: uns = souog{ 





SxACP Guujes,doo § : SpUNDG 50 9NO {II ||| 
a SIQ40ANNG oe NIOHIOON ial J02Uu05 C) PUIAA 
‘ Wj8d a” “ ~ ” Y id il 9 an Po cea oe 99s4 a Wy QA > 9104 RSs ‘ SS 
“18910 UI IND) BP/YMOION , “~~ ~~ Aspunog uoneiBed oso” ped sja10u09 0 W614 usiuid (S) dial Bap ¢s¢ 
UR d PURSIPUL 0" 8. oe ‘ Sul PUOUTSOL | NY) sajod suoudajo, «© = 1184 > YMOLBu9puR|l|||| AMS ~ J2 Yeah 
(poKede pec ae ee vie SA Wd A punoi6 usdo » Mais ay Apno|9 
POY POAC wm inowod § OF 9/90 9S|N - puno/6 Apues Uedo BR — ——_— 
N91) doar) “= ae a“ ANOWOIB 0s“ \ Uojss 9idop AO|PUS SulPpING gay pusbay PMY Jaquny juedioiyed 


Figure N.47. RW4 Planned Route 
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Figure N.48. RW4 Executed Route 
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Figure N.49. RW4 Wheel Test CP # 2 Figure N.50. RW4 Wheel Test CP # 4 





Figure N.51. RW4 White Board Test 
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Figure N.52. RW5 Planned Route 


syeraye ¥) 


33 Ob JEAIORN) ANOQNED 
e00's:s He0g den 





Ueid a}NoY pajsnipy sjoalqng=—=— 
ayNoy pauueld sjoalqng—< 








s1919M 000} 090 «= (000s—s—iCiS (OsC(iéi LC (CLC (GC CSC“(«éi DCCC Ci EC(‘ié‘i:SC*«CSC(<té‘itSC(‘i‘iSCCt*«iD o os 
punosS uedo yBnoy RS q 


che} YSIS =— SOLE «HENS 











YUIRY ot: umi-yo04| | 
@axAep Buyules;doo1, =} ~- $punog jo yn9 || ||I| 
. ; 490}}00% Uns MOIS - O10 4 PS DUIAA Yaa = 
nedae  ~ | 7 YO 20 ANNO ae nom - 8010 5 ME josju0> C) ! 
“480103 UJ IND) OPI YMoOUON | a a a ApUNDG UOAdIB9A 4. ied viste> jot usjul4 (0) ‘dweo] bap 09 
UP d PURSIPUY _-°" **..,-° Bull eucudeles Jp Sojod ouoydejay ¢ = 118M * YAOII9pUN | ||| AIS ~ Jaujes 
(peagdun) pooy (4n0juod y §) oul] wUO4) raieg fais punoBuedg. vas V $ 1893/9 4 M 
por doar, “~ snowo2 805. \_”_vorssoudap MOUS SulpInd amy puabey SMu JOQUUINN Juedioped 


233 





BEBI AeW 

{VSN Ja61a09 "Yy UOWIS UReNded Aq paydaud pyjaij 
YSN sanueg WeIIIIAA J0fe |W Aq dew 

eUOpeD ‘PO i 


sf tee 
Meigs ee i 


Op /: ey 
Naf z 
* eee 
he aw 
> 







peoy bui\6i9 


pasidxy awl,  ® 
aynoy MAN syajes yelqns @ 
a}noy paynsaxy s,joalqns ~~ 
JOWUOW Aq UOHeJUSUO yOalqns yw 
spunog Jo nO 
yoIBUD SSeEdWoyD »¥ de 
yoayD deyy 
yoeyuy ssedwoy 


Figure N.53. RW5 Executed Route 
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Figure N.54. RW5 Wheel Test CP # 2 Figure N.55. RW5 Wheel Test CP #4 





Figure N.56. RW5 White Board Test 
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Figure N.57. VE1Planned Route 
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Figure N.58. VE1 Executed Route 
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Figure N.59. VE1 Wheel Test CP # 2 Figure N.60. VE1 Wheel Test CP # 4 





Figure N.61. VE1 White Board Test 
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Figure N.62. VE2 Planned Route 
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Figure N.63. VE2 Executed Route 
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Figure N.64. VE2 Wheel Test CP # 2 Figure N.65. VE2 Wheel Test CP #4 





Figure N.66. VE2 White Board Test 
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Figure N.67. VE3 Planned Route 
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Figure N.68. VE3 Executed Route 
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Figure N.69. VE3 Wheel Test CP # 2 Figure N.70. VE3 Wheel Test CP # 4 





Figure N.71. VE3 White Board Test 
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15. VIRTUAL ENVIRONMENT PARTICIPANT NUMBER 4 
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Figure N.72. VE4 Planned Route 
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Figure N.73. VE4 Executed Route 
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Figure N.74. VE4 Wheel Test CP # 2 Figure N.75. VE4 Wheel Test CP #4 





Figure N.76. VE4 White Board Test 
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16. VIRTUAL ENVIRONMENT PARTICIPANT NUMBER 5 
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Figure N.77. VES Planned Route 
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Figure N.78. VE5 Executed Route 
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SUBJECT DID NOT 
REACH CP #4; 


NO DATA 
COLLECTED 





Figure N.79. VES Wheel Test CP # 2 Figure N.80. VES Wheel Test CP #4 





Figure N.81. VES White Board Test 
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APPENDIX O. RAW DATA 


1. GENERAL INFORMATION 
Participant data is referenced by the participant identification (ID) label (A — 
Assistant, M — Map, P — Pilot, RW — Real World, VE — Virtual Environment). The 
number corresponds to the participants internal group label. Data fields that are left 
blank represent information not recorded because a participant did not undergo the test or 
failed to reach that point in the course. Pilot participant data and assistant data is utilized 
for questionnaires only. The course data varied due to changes in experiment 
methodology for some of the pilot participants. Empty data fields represent information 
not recorded because the participant failed to undergo that portion of the experiment or 

did not wish to answer the question. 

ID Abili Time 

AR/CAV 
21-Jul 


| RW1__| Real World| 34 | M_| 04 | Army | SC__| _Intermediate__| 29-May | 13:00_| 
| RW2_| Real World 
Lis 
__RW4 | 
fo 








/Real World. 
/Real World] 37 | M | O04 | Marine | AV | Intermediate | 16-Jun_| 13:00_| 
3 

| 03 | Marine | IN | Intermediate__| 20-May | 8:00 __ 
Marine | MI_| Beginner | _3-Jun__|{_ 12:30 _| 

38 | M | 04 | Marine | FA/MI | Expert | 15-May | 8:00_| 

| M_| 04 | Marine | IN | Expert | 15-May | 13:00_ 










yes 


2. INITIAL TESTES AND QUESTIONNAIRE RESULTS 
The initial tests and questionnaires are in Appendix E. Answers for the Map Test 


are located in Appendix E. 1. 


Participant Bar Santa Santa Santa Banker 
ID a Evaluation | Barbara Barbara | Barbara os aan Ability 
Score Raw —— a Grou D Grou D Grou D 


eon 
[1] Expen [a6 [3.07 igh | 92 [High iemediae 
Intermediate 
| 
Cc Silo i i ia he See 2 ee 
ete Sa ae. Flee Low 
eS hE ar eee eee ee 
aa eee ae 24s | ee 


3. ROUTE ERRORS 


The data provided in this section consists of the map checks, errors, error 

























distances, and route leg classifications. The data appears in its raw form, summations, 
and normalized form for each of the experiments fifteen participants. The codes for the 


utilized are listed in Table O.1 and in the List of Abbreviations (pp N 


Table O.1. Route Errors Abbreviation Table 













Dawa 


a. Route Data Summation 


Controls Control | Landmark | (Banker) (Lisp) 
Attempted Found Score Average Average 
Planned Planned 

Route Route 
Difficulty | Difficulty 

Level Level 


re eee Eee: es 
| 2 2S ee oe oe ee 
ee eS ee 
Pd |B eee 2.33 ee 






















Participant| Errors- Dist-Tot | Dist Per | Normalized Normalized Normalized 
ID Tot Error Error Error Error 
Score/Attempt | Score/Found Score/Found 


3260 
20.82 20.82 






A 
127.6 14.18 
3690 105.43 128.46 
809.25 202.31 269.75 213.56 
167.38 200.85 240.73 





l 


A EO OE EO ee ee ee ee ae 
P VES | 6 50 4540 756.67 | 189.17) 252.22 157.89 


l Onl 





3), 










| Controls Landmark 
Map | Map Check | Map Check | Map Check | Attempted| Found Score 


Pi See eee 900 
2 ee OT OT le 900 
Re a dA le 9200 
Pia) os ae le ee ee ee 
LS i eee ee ee ee ee ee 
RZ 2s 294 oa 294 sk ao 
BEE | OE ae nO | 0 8] 8... 0 
iss (oe ee ee ee a 
A 
_ vel} 9 | —7 9 __{ 9 | 9.00 
eS ee Oe eee ee as 
Pe ae I eee ee ee ee a 
a 
| er | 3 |S Ss as Se) eee 

















b. Route Data Leg SP to CP1 


Parti: :pant Errors Total | C-1 | M-1 | MC-1 | MCL-1 | OB-1 | New Rt-1 | Checks Control 
a 'P ~ aa = ee ae SP-1 — SP-1 
a 
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c. Route Data Leg CP1 to CP2 


Participant| Errors C-2 | M-2 | MC-2 | MCL-2 | OB-2 | New Rt-2 | Checks Control 
ID 1-2 pia] ae 1-2 en 1-2 


Ae | ae Eo 

Pe 6 i aoe eee a 

be M3 ig) | 50 io oe ee ee ee 
my? Pit = 








Zz) 


e. Route Data Leg CP3 to CP4 


sia Errors Total | C-4| M-4| MC-4 L-4 | OB-4 | New Rt-4| Checks | Control 
3-4 Dist-4 —— 3-4 ——_- 3-4 
—2— 






2 Meee, 305ee|/ One| 0 
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g. Route Data Leg CP5 to CP6 


articipant | Errors Total |C-6 MC-6 | MCL-6 | OB-6 | New Rt-6| Checks | Control 
res ———- an ee re 5-6 | 5-6 





1 
ee a ls a ee a a 


h. Route Data Leg CP6 to CP7 


—s Errors | Total | C-7 MC-7 L-7 | OB-7 | New Rt-7 | Checks ell 
=. ee en 6-7 a 
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i. Route Data Leg CP7 to CP8 


la Errors | Total M-8 | MC-8 L-8 | OB-8 
7-8 Dist-8 


Pon i ion oro Fo fo 





New Rt-8 


be Control 
Score 7-8 | Found 7-8 









ie 
tw 


L a is 
= 
| ao ay 
a 
| Sse 
| a a 
—_ 


MC-9 OB-9| New Rt-9| Checks | Control 
oa Se a 8-9 —a 8-9 
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k. Route Data Non Error Checks 


Participant| C-Non MC-Non | MCL-Non Checks Score 
ID Errors ares — Errors Errors Errors - Non Errors 
a Se a 0 el a ee 





l. Route Data Totals 


On -Tot Per a an ——— 
oe a a eo 
vice Sess Mimo | 0F | 12 om |_| ao eee 





















a ee 
ee ee ee 
a | 

| fC ae 

ee as isso Oi ec a 
BeRwa | 97) i) 4053. eo | oe a or a ee 
Ed a ae ee a ee ee ee 
hee" eee ee OR) OT eet Se 
Rn a EN a a | ae 
Pee Se Pe) ECO ee ee eee eee See 
ee EE es eee ee OT a ae 
iso aes en ee ee a 
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m. Leg Difficulty Evaluation Banker 
Each leg is evaluated utilizing MAJ Banker’s Route Classification 
(Appendix L.2). “B” stands for Beginner, “TI” stands or Intermediate, and “A” stands for 
Advanced. The total is based on a point value system of B = 1,1 =2, and A= 3. The 


Average is the total divided by the number of legs. O-1.50 is an average course difficulty 


of Beginner, 1.51-2.5 is Intermediate, and 2.51-3.0 is Advanced. 


TRwi |i [alifalir1{[a|a|[al|a || 267, 
omnjeel [ee | OM Lave wieeleeam | ved oat [eeAwn| manele 244m 
ee SE ee ee ae eee a eee 
es a Le A | | ADA | CALA ae 
2 See ae Ea eA ee ee eee 
a Ne | P| P| ON) | oe Bicel|B.| A] ws. | eae a 106 Z| 
ee er a ee ee ee ee ee ie ee 
a EE BE BAB | BBA a See 
Le, | Se Ee ae 
ee | A | ey AS | FIA is) | 0c 





n. Leg Difficulty Evaluation LISP 
Each leg is evaluated utilizing LISP Programs Route Classification (Appendix 


L.3). See above for code definitions and summation specifics. 


<a ee bes Si jr Oe ge 
se a a os 
I I 







ee led | eae va 20222 
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4. WHEEL TEST RESULTS 
a. Wheel Test Results for Control Point 2 


eA esa) -31_ | eae 30.33 






—w2—[south | 12 [78 [229] 166 [8] 1] a9 
| M3 North | 16 | 77: | 248 | 147] -7— | 8] 23 T1267 
| M4 | West | 43 41 8320] 29 | 57_—| 50 4s 
LS  giisoutta: | 405 Bie 207) 45) 4839" | es | 
Lael Blsouth Wise ivan Zoaii3i|7=6) |) 36 || 5 eto | 
Pw? Bsoccn a esai2a0 a5 | 0 | 2 ae 
Los Escrito 20a en St 34 | 3 aa 
LRA Gwese 2952) 16a) isan ie) 77 82 | a | 
PEW 5. Si ENodhy Is) | Tee eac | -3| )  aee 
PEEL 8) Acrows | Oe Be | een 491 Nee| EB | -9 |) ae 
Pie? Eso) Bo |86))237)|29) Peo | 3 41 | aoe 





| VeE4 || South | 40 74 | ots [ies 4 22, eT) 





b. Wheel Test Results for Control] Point 4 and Total Wheel Test Angular Difference 
Total Wheel Test Angular Difference is the value of the Average Angular 
Differences for CP2 and CP4 divided by two. No data was collected on four individuals 
(M5, RW4, VE2, and VE5) because they failed to reach Control Point 4. Their Total 
Wheel Test Angular Difference is the save as the value for the Average Angular 
Differences for CP2. 


Participant Time | CP1|CP6;\CP8| Diff | Diff | Diff \|Ave Angular| Total Wheel 
ID (sec) CPI CP6 | CP8 Diff CP4 | Test Angular 
Di 


| Mi | South | 38/86 | 227 | 153) -28 | 8 37] 24s | 2733 
yO Cree ee eee 
De Nan i 697 [1252 eet -11 | 17 ese 267 TT aee7 
| M4 | South | 47 | 49 | 187] 110] 9 | 48 | 6 | 2100 | 33.17 | 
Cook 2 ee ee ee 
| RWi_ | South | 28 | 74 | 230[ 182{ -16 | 5 | -66 | 29.00 | 31.00 
| RW2 | South | 24 {57 | 240] 180] 1 | -5 { -64 [| 23.33 | 19.00 

ic ee eee Se ee ee eee ee 
|_VE] |Arrows| 62 | 30 | 168] 110{ 28 | 67 [| 6 | 33.67 | 21.17 
eS a a a a a as ee ee eee 
es Teel ee ie 226 |e ee a Be Oe 
| VEA | EasyN| 60 | 89 | 269] 180] -31 | -34 | -64 | 43.00 | 26.83 
Ee ee ee ee | eee | ee 
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5. WH ~¢ BOARD RESULTS 
a. White Board Normalized Distance Differences from Actual Normalized 


Distances 


SP to | CPI to| CP2 to| CP3 to| CP4 to| CP5 to| CP6 to| CP7 to| CP8 to| CP9 to Vg OB 
CPI | CP2 | CP3 | CP4 | CP5 | CP6 | CP7 | CP8 | CP9 | SP 
“al at 
Dist | Dist 


|-0.01 | -0.01 | -0.01 | 0.03 | 0.03 | 0.06 | -0.04 | -0.01 | 0.03 | -0.08 | 0.33 _| 0.033 | 
Te 
| M3_| 0.02 | 0.00 | -0.01 | -0.04 | 0.03 | 0.02 | 0.00 _| 0.06 | -0.06 | -0.01 | 0.27 | 0.027_ 
| M4 | 0.00 | 0.03 | -0.01 | -0.03 | 0.02 | 0.09 | -0.01 | -0.05 | -0.04 | 0.00_| 0.28 | 0.028 | 
| MS_| 0.02 | 0.00_| -0.03 | 0.02 | 0.03 | 0.02 | -0.01 | 0.01 | -0.03 | -0.03 | 0.19 | 0.019 | 
CRWI_| 0.00 | 0.03 | 0.00 [ -0.05 | 0.01 | 0.04 | -0.02 | 0.03 | -0.04 | -0.01 | 0.22 | 0.022 
CRW2_|-0.01[ 0.01 | 0.01 | -0.03 | 0.05 | 0.03 | -0.02 | -0.01 | 0.00 | -0.03 | 0.19 | 0.019 
| RW3_| -0.03 | -0.01 | -0.01 | 0.00 | 0.02 | 0.04 | 0.02 | 0.04 | -0.06 | 0.00 | 0.24 | 0.024 

3 

2 










_RW4 |-0.01| 0.01 | 0.01 | -0.03 | 0.05 | 0.06 | -0.04 | -0.03 | -0.04 | 0.02 | 0.30_ 
RWS [0.02 | -0.04 1 0.01 10.00 10.01 1 0.08 | -0.01 10.00 | 0.03 










b. White Board Angles 


| M3_| 125.11 | 141.25 | 130.59 | 96.82 | 96.61 | 109.81 | 90.87 | 169.30 | 166.80] 15.28 _ 
| RW2_| 178.66 | 178.82 | 122.12] 111.69 | 90.88 | 99.38 | 120.00 | 149.72 | 140.01 | 48.87 | 
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c. White Board Angles Differences from Actual Angles and Totals 


SP,1,2| 1,2,3 | 2,3,4 | 3,4,5 | 4,5,6 | 5,6,7 | 6,7,8 | 7,89 |8,9,SP|9,SP,1| Total Avg 
WB WB 
gs ae 


PMI} 27.641 8.09 11.91 1-31 7150-78} 6.29 {59.741 22.39 9.04 | 26.26 | 229.87 | 22.987 
| RWS5_| -7.94 | 16.55] -3.92 |-44.31|-42.66]-23.92] 41.16] -0.99 |-40.30] 18.23 | 239.97 | 23.997 _ 
[-13.77] 22.18 | -39.35|-58.91|-39.42| -6.69 |25.25| 2.36 |-25.98] 12.06 | 245.97 | 24.597 _ 





















6. UNPLANNED ROUTE RESULTS 
ee ee a 
We to CP4 to CP4 min seman Route aw Route 
CP8, W to CP4 
cman 200m {Trail N to 6: A 
clearing, W to 
CP4 
Eee 
trail toward CP6 
& E to CP4 
oe 
——— + 
— -_ . 
a ee dee | 
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c. Miscellaneous Questions 
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e. Interface Questions 
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f. Descriptive Statistics for Debriefing Questionnaire 


The following table displays summary of the descriptive statistics for the 
questionnaire minus the data for the pilot participants. The total number of participants is 


15. Missing data is from questions that were not administered to real world and map 


group participants. 
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CourseQ1 
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MiscQ3 
MiscQ4 , 
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Model Q1 
ModelQ2 
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ModelQ5 
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g. Model Needs 


For the each item “1l’’ means the items does need to be included and “O”’ means 
the items does not need to be included in a model used to prepare an individual to 


navigate through the actual terrain. 


1) Buildings 
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2) Miscellaneous Objects 





3) Obstacles 


pits/fox holes | shallow ditches 
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4) Roads 
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h. Top Six Model Needs 
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APPENDIX P. ENVIRONMENT COMPARISONS 
1. GENERAL INFORMATION 


This appendix outlines the differences in the training conditions and participant 
results between the Banker and Goerger experiments. The training conditions include a 
non-real time model, real time model, real world exposure, and map only study. NA 
indicates characteristics that do not apply to a training condition due to the nature of the 
training medium. Results are broken down by experiment and by training condition. 
Results such as the Total Map Check Scores, Wheel Test Angular Differences and White 
Board Angular Differences are not directly comparable because of the different 


information included, means of measurement, or tests conducted in the separate 


experiments. 
2. ENVIRONMENT CHARACTERISTICS 


The following is a listing of characteristics and which training treatment 


Model (Banker Goerger 
Yes___| Yes 
Yes 


implements each one. 
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Characteristics (continued) Non-Real Time Real Time Model | Real World 
Model (Banker Goerger 


Yes Yes Yes 
Yes 
Yes Yes 
EO seee NOs Nes el Ng 
Realistic Control Point Markers 
Realistic Telephone Poles 





Shallow Ditches | 
Sound 
Yes 
Yes Yes 
Yes Yes 


Table P.1. Training Environment Characteristics 


2/74 


3. PARTICIPANT GROUP RESULTS 


Behavior and Performance Non-Real |Real Time| Real Real Map Map 
Differences Time Model\| Model World | World |(Banker)|(Goerger) 
Banker Goerger) |\(Banker)|(Goerger 


Average Control Points Found 
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Average ErrorsPerCP#4 | 0.600 | 2.400 | 0.800 | 1.600 | 1.000 | 1.200 _| 


435.789 | 627.040 464.556 | 345.684 | 502.850 


Distance Per Error Per Control Point 51.880 131.984 | 61.898 | 80.046 | 46.714 | 83.036 
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ge Map and Compass Checks 000_| 5.800 


0.258 0.000 | 0.806 0.026 


20.134 27.668 26.666 

31.890 19.418 23.918 

Average Wheel Test Angular 26.166 NA 27.500 NA 26.634 
Differential (Pointing Task) 

NA 27.588 ype 


Average White Board Test Angular NA 21.376 NA 
Differential (Geo target Placement 
Task) 


Average Unplanned Route Execution 


Table P.2. Participant Results by Training Treatment 
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APPENDIX Q. NAVIGATION CYCLE 
1. GENERAL 


The phases of the navigation cycle are displayed in Figure Q.1. This diagram is a 
simplified flow chart depicting how an individual may conduct navigation through an 
environment. It is generic in nature to allow it to be applied to navigation of varied 


media and environments by any mode of locomotion. 


Navigation: 


Static Imagery 
- Compass Imagery 
- GPS 
- Determine Position & Orientation - Maps 
cia tie - Photos ; Mg 
- Determine Method of Movement - Video 


- Develop Strategy . aoe - Other Sources 
- Su 


- Other Sources 


- Environment 


Coarse 
- Recognize Error Movement Movement 
. Develop oe ad tu “ VEpeRene - Visualize Target Arez 
reacquire position - Search Area 


. : - Conduct Movement : 
and orientation rane Be HERON - Filter Information 


Maintenance 
(Mental & Physical Maps) 


- Linear and Point Objects 
- Distances/Scale 





Figure Q.1. Navigation Cycle 


2) 


The model takes into account many of the factors used in previous models which 
attempt to depict the process of wayfinding [JUL 97] [WICK 98]. Figure Q.1 is similar 
in its purpose and design as the one currently under development and verification by 
Dr. Wickens [WICK 98]. However, this chart is more generic than Dr. Wickens’ and is 
designed to quickly identify the stage of navigation an individual is in during course 
execution. Using the data collected in this experiment in conjunction with a viable 
diagram outlining the task of navigation, we can gain a better understand of the process 
of navigation, determine where failures in navigation are most likely to occur, and create 
systems or provide training to correct those shortcomings. With proper training and 
evaluation tools, we can predict trouble areas during mission planning and preparation 
that can be corrected to assist in conducting more efficient maneuvers to target areas, 
reserving resources for other crucial requirements. 

The diagram separates the task of navigating into four distinct areas; route 
planning, route navigation, navigational aids, and error recovery. While navigating, all 
individuals will undergo route navigation. The phase of error recovery is a tributary 
phase that is utilized only if individuals feel or recognize they are no longer following 
their initially visualized or planned route. 

2. ROUTE PLANNING 

Before movement is conducted or after a navigational error has been recognized, 
most individuals fix their position and orientation, identify the intended goal, determine 
method of movement, and plane route to traverse the space between the current position 
and target position. These processes are grouped together and are known as route 
planning. To facilitate route planning, individuals seek information about the 
environment and their location within that environment from navigational aids (Appendix 
Q, Section 5). 

3. ROUTE NAVIGATION 

During route navigation, individuals break their movement down into three 
distinct areas, coarse movement, fine movement, and maintenance. Coarse movement is 
categorized as the general movement that occurs as individuals traverse indistinctive 
terrain in search of a linear catching feature or landmark which tells them they are in the 


general area of the target. Distances vary in length based on the limits of visibility and 


Vis. 


target area size. The area covered during movement provides few cues which individuals 
focus their attention on. Individuals are concerned with directional signs [PASS 84] that 
tell them they are moving in the appropriate direction. They are also concerned with 
distinctive features or regional signs that will tell them they are on their planned route, in 
the target area, or have passed the target area. For example, if traveling from New York 
City to Chicago by car to watch a Chicago Bulls game, we are not concerned with the 
road signs telling us that there is a sale at Johnson’s Lumber. However, we are concerned 
with signs telling us “Welcome to Ohio”, “Interstate 90 North 2 Miles”, and “Mississippi 
River”. The first two provide us with information that we are traveling in the right 
direction, directional sign, or that we are close to our target area, regional sign [PASS 
84]. The last sign tells use we have gone past our intended destination and we need to 
stop and plan a new route to Chicago. In a natural environment, directional signs for 
dismounted movement would be items such as the sun, moss on the north sides of a tree, 
the North Star (Polaris), or the flow of major rivers. Regional signs would be clearings, 
sand dunes, red wood groves, or villages. 

The fine movement phase is characterized by a more detailed search using more 
distinctive landmarks. During this phase, individuals pay more attention to their 
surroundings focusing on minor changes in the environment known as identification 
signs [PASS 84] that may lead them to the target. An example is when we reach the 
outer loop of the Chicago beltway, we start searching for specific exits and street names 
to take us to the stadium. As we get closer to the stadium we search for information 
about parking. Identification signs for this experiment would be the wooden shanks, 
pavilions, telephone poles, rock piles, and major trail intersections. Due to the random 
and relatively unstructured nature of natural environments, any unambiguous terrain 
feature can be used as an identification sign. 

Throughout the route navigation process, individuals regularly perform mental 
map maintenance. As individuals maneuver through their environment, they continually 
search for information from navigational aids (Appendix Q, Section 5) to confirm general 
location and proper movement heading. This information is known as reassurance signs 
and includes items such as mileage markers and exit numbers [PASS 84]. They are 


usually selected during the route planning phase and used as self-imposed checks during 


PUFFS) 


route navigation to ensure individuals remain on their planned route. In a natural 
environment, reassurance signs are more ambiguous than those found in man-made 
environments. For this experiment’s environment, reassurance signs would be the 
number of trails crossed, trail intersections, high ground, low ground, or power lines. If 
there are differences between the physical or mental map and the information the 
individual perceives, the differences must be resolved. If there are no differences 
between the mental map, physical map, environment, and where individuals believe they 
are, movement continues toward the intended destination. Individuals freely move 
between coarse movement, fine movement, and maintenance based on the complexity of 
the route and mission. 

Resolving minor differences can be as simple as updating one’s mental map to 
include the new Pizza Hut built off of Exit 68 or the burn barrel south of Control 
Point #2. When they are unable to correctly resolve differences between their maps and 
the environment because they failed to recognize a landmark, they become confused. 
This is a phase where individuals attempt to resolve major differences in the mental and 
physical maps with their surroundings. If they cannot determine if they are lost or if they 
need to update their mental map, individuals remain in a data collection loop attempting 
to gather detailed information from navigational aids to make that determination. At the 
conclusion of this phase one must return to route navigation or move to error recovery. 

4. ERROR RECOVERY 

Once individuals have identified that they are lost, they enter the error recovery 
phase where they determine what their mistake may have been. Determining their error 
assists the individual with coming to grips with the situation and provides an indication of 
a potential recovery strategy. After recognizing the error committed, individuals move to 
the route planning phase. 

5. NAVIGATIONAL AIDS 

Navigational aids are used to provide individuals with information to update their 
mental maps, determine position and orientation, plan routes, and execute movement. 
They are utilized in accordance with an individual’s experience, training, and confidence 
level. The aids are placed in one of two subcategories based on the type of information 


they provide, static imagery or dynamic imagery. Static imagery renders propositional 
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information about the environment and the individual’s place within the environment. 
This imagery is provided from items that furnish positional information (GPS, map, 
signs), orientation information (compass, map, sun), or stationary target information 
(map, pictures). Dynamic imagery supplies temporal information about the environment 
and the individual’s place within the environment. This imagery is provided from items 
that furnish disambiguating or continuous information. This information is derived from 
the environment, VEs, videos, and other active sources of information. This information 


is used to disambiguate positional information. 
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